Read-only system improvements

Massa Improvement Proposals
1

This proposal relates to improving the read-only system in the Massa node.

Needs

  • Simulate an execution to estimate the gas usage of a call/executeSC
  • Simulate an execution to estimate coin costs
  • Call a read-only SC function to read its result for free
  • Automated smart dapp backend testing (eg. multi-SC)
  • Simulate an execution for debugging
  • Simulate an execution sequence as part of IDE tooling

Currently the system is limited:

  • issues with nested calls for non-existing addresses
  • no easy way to estimate coin consumption
  • no return values from read-only executions
  • gas inconsistencies
  • no automated testing features / limited debugging capabilities , especially multi-contract
  • no way to atomically execute multiple read-only runs at the same slot

Solution

New endpoint

  • read_only_sequences([[ReadOnlySequenceRequestItem]]) → ReadOnlySequenceResult or Error
  • ReadOnlySequenceRequestItem is an enum of different possible requests
  • ReadOnlySequenceResult { execution_slot: Slot, items: [[ReadOnlySequenceItemReturn]]}
  • The inner array states a list of tasks to run consecutively in the same temporary context
  • The outer array states allows running a set of independent task lists at the same slot (context is reset between each item)

Supported requests

  • get_balance { addr } returns { addr, balance }
  • get_bytecode { addr } returns { addr, Option }
  • get_datastore_entry { addr, key } returns { addr, key, value: Option }
  • has_datastore_entry { addr, key } returns { addr, key, bool }
  • list_keys { addr, key_start, key_start_included, key_end, key_end_included, prefix, max_count } returns {addr, keys: [byte]}
  • set_balance { addr, balance } returns {}
  • set_bytecode {addr, bytecode} returns {}
  • set_datastore_item {item, key, value} returns {}
  • del_datastore_item {addr, key} returns {}
  • execute_operation {call_stack: [address], op: Operation} returns {outcome: ExecutionResult, state_changes: […], return_value: [byte], gas_usage}

(limited total item count and gas for security)

Usage examples

Estimate gas and coins simultaneously

  • set the coins available to the sender address to a high value
  • make sure the SC reimburses excess coins
  • run the operation
  • count how much gas was spent, how much coins were spent
  • recover the output value

Simulate a full dapp for integration testing

  • spin a local sandbox node or use buildnet/mainnet
  • simulate the deployment of multiple smart contracts
  • set the right initial conditions
  • run a sequence of calls on the different smart contracts
  • read the output state
3 Likes
2

My thoughts on the needs and proposal

Let me add some details on the needs:

1. Accuracy of gas estimation

Issue: #4742
Unless proven otherwise, this looks like a bug that should be fixed in the current endpoint.

2. Simulate an execution to estimate coin costs

This would indeed be very convenient.
I think this feature could be added to the current read-only executeSC & callSC.

  • Storage costs can be computed by looking at ledger changes.

  • MAS transfers should also be taken into account.

  • The “coin cost” could be positive, negative, or even null if it cannot be computed.

This could be achieved by simply adding a new field in the returned object (next to gasCost).
That makes it almost non-breaking.

3. Call a read-only SC function and retrieve its result

Issue: #4913
Currently, executeSC in read-only mode does not return the contract’s value.
For me, this is just a missing piece in the implementation. It should behave like readonly callSC (which already handles it correctly).

This is also a non-breaking change.

Related issue for write mode (callSC & executeSC)

#4767
It would be useful to include the returned value (e.g. the result of a return instruction) in the operation info.
Examples of usage:

  • retrieving the address of a deployed smart contract without relying on events.
  • Using readonly executeSC to do multicall ( to gather result of multiple readonly calls in one call)
    Again, almost non-breaking—just adding a field.

4. Read-only calls when only reading on-chain data

Issues: #4912, #4677
We sometimes face issues when we just want to read on-chain data.

Suggestion: add a new parameter (e.g. ignoreBalance) to read-only calls.
This would skip all balance-related issues (storage costs, account creation, nested call coins, etc.).

Again, this is almost non-breaking.

5. Transition strategy

Fixing or extending the existing endpoint would make the transition much easier:

  • No need to integrate a brand-new endpoint in tooling.

  • No need for deployed dApps to choose which endpoint to use.

6. About testing & IDE tooling

The points about automated backend testing, debugging, and IDE integration are important, but in my opinion:

  • Even with the proposed solution, building a full SC test framework or IDE plugin would be a huge amount of work.

  • Putting such functionality directly in the node raises concerns:

    • Extra workload for the node

    • Increased attack surface

  • From a usability perspective, relying on an online RPC for unit tests or IDE workflows feels awkward.

The SC testing topic is big, and we haven’t invested much in it yet.
It might be better to improve the current tools first before trying to push these responsibilities into the node.

3

Nice writeup @peterkaj and it does provide a solution that is more minimal and plug and play than the one proposed above. Let me refine your ideas.

Accuracy of gas estimation

OK it’s a bug, let’s fix it, but without breaking stuff. The issue is that given the bug, many apps are already multiplying the estimate by a constant factor to get a better estimate. Some of those apps might break if they end up overestimating after the fix. Maybe we can just add a new field “gasCostV2” that provides a better estimate without breaking previous software. What do you think ?

Simulate an execution to estimate coins

If I am not mistaken, the changes caused to the ledger by the read-only call are already returned. That way, knowing the balance of the caller addr before the call, you can know how many coins were actually spent, assuming the SC returns any excess coins back to the caller.

The only issue would be atomicity: you first need to query the amount of coins the caller has, and then you will see the new balance in the list of changes. If spending happens in-between the two calls, the measurement will be wrong.

Therefore, adding a field “callerMasSpending” as you sugested makes sense. But there is a better option with the `simulateInitialCallerBalance` as described below.

Call read-only function and retrieve its result

This is already implemented for read-only calls here. I agree that we could add it for read-only execute bytecode as well. Let’s just be mindful of this.

Read-only calls when only reading

Technically, if there is only reading happening, there should be no coin spending or storage writes, because there are no writes happening.

That being said, in case we want to be robust to writes happening in a read-only context, it will be simpler to add an optional field `simulateInitialCallerBalance`. This would allow force-setting the caller’s balance to a given value before the call, and everything else remains unchanged. Setting it to a high value can fill this need. It also feels the need mentioned in the Simulate an execution to estimate coins section, because the initial balance is atomically known, and the read-only operation returns ledger changes, so everything can be deduced atomically.

SC Testing

I agree that this deserves its own separate discussion.

4

About Accuracy of gas estimation, i don’t see anything breaking.
The case where estimation goes over the max allowed should already be handled.
It is the case for all dApp using massa-web3.
For me, it is better to go with the current existing field.

About Simulate an execution to estimate coins, your proposition is to do the maths on the client side (massa-web3 for instance) using the ledger changes that are already returned and “mocking” the caller balance. I’m ok with it!

And agree for the rest of your propositions!

1 Like
5

Ok then let’s do this:

  • fix gas estimation for read only executions
  • add optional simulateInitialCallerBalancefor both read only endpoints
  • add read-only executeSC return value (if any / optional)
1 Like