- Meeting Notes
- Need For Indexing/Overview
- The Graph
- UI Decentralization
- Decentralized Compute
- Custom Indexing Solution
- The community seems to be leaning towards a custom solution. More exploration and research will need to be done, but that does seem to be the way forward.
- In the meantime, a more limited subgraph implementation could be helpful to buy enough time to get the indexer solution right.
- There could be a more short-term product created that is a proof of concept for the custom indexer.
- Team code review of Tractor will be necessary to have some sort of interface lock in order to start making indexing decisions. Some of this be done in parallel so that the contracts functionality can be adapted to accommodate the needs of the indexer if necessary.
Need For Indexing/Overview
- One of the fundamental principles of building on a censorship resistant immutable chain is that you want to have the on-chain data remain as light as possible. To achieve that, it requires a lot of indexing of the on-chain data in order to process it off-chain. In order to understand or interpret potentially very limited data encoded on-chain, each system of smart contracts typically requires its own indexing software.
- It is important to consider how the indexing layer factors into the censorship resistance of the systems being created.
- The Graph is a network to host indexing software in a decentralized fashion. There are multiple sub-graphs which are indexers for Beanstalk and the Bean token that run on the Graph network and are hosted in a decentralized fashion now. The bean.money website uses both of those sub-graphs and the website is currently hosted on a centralized server.
- The centralization vector associated with using Beanstalk has been minimized as much as possible through the open sourcing of the website code, and through the deployment of the sub-graphs on the Graph network.
- The hosting of the website makes it such that there's still some need to interact through some centralized intermediary. In theory, since the website is open source, you can run it all locally and having the data indexed on the sub-graph makes doing so much easier. But from a centralization perspective, the vast majority of users are interacting with Beanstalk through a centralized host.
- The question that must be asked is until there is some sort of decentralized hosting network or way to host websites in a decentralized fashion with a competitive user experience to centralized hosting options, what is the value of the labor intensive work of developing subgraphs. Is it a better idea too instead develop centralized indexing software that runs on a centralized server and open source that code such that anyone can run their own copy.
- Subgraphs have the limitation that they can't index multiple chains, while Tractor orders can be placed on any compatible chain.
- If we develop our own decentralized indexing software, it would have similar centralization vectors as is currently the case with the website.
- There is a separate question of what is the actual indexing work that needs to happen.
- Eventually there may be off-chain zero knowledge solutions capable of indexing Tractor's entire order book.
- Whatever tech is chosen will entail a commitment to using that tech for a substantial amount of time. It's important to choose the best solution that will minimize the amount of code that needs to be replicated.
- The Graph has its own network with independently operating nodes, similar to the architecture of Ethereum.
- Indexers for The Graph stake some tokens and execute the instructions to do the indexing. After they submit proofs, they are subject to slashing risk if their work is incorrect.
- The Graph implements a very specific type of indexer which imposes engineering constraints that can affect the products that can be built. It may or may not be the case that The Graph supports everything that we need long term.
- If The Graph doesn't support something we need to do, at that point we could consider forking it to provide some upgrades or starting from scratch to build something optimized for Tractor and Wells.
- The Graph offers a centralized host, which is an instance of their software running on a cloud provider. The software is open source, but too demanding for most people to run themselves. Other pieces of the infrastructure, such as the UI, can be run on most modern computers.
- Ultimately if we can make it easy to run the different components, that is a de facto form of decentralization. Even if it is not on a decentralized host, if it can be run by users themselves that produces a similar result.
- It is not currently possible to have a subgraph run on a locally forked version of Ethereum for development and testing purposes due to the volume of queries required to index all of Beanstalk over its history.
- Decentralization is a matter of the least common denominator, such that until there is a way to sufficiently decentralize the UI, the value of decentralizing the indexer is marginal. Open sourcing the indexer is incredibly important, but as long as Beanstalk Farms is hosting the UI, there might not be much benefit to having the indexer deployed on a decentralized system as opposed to running it locally.
- The UI has been designed to be runnable as a static website so it can be hosted somewhere like IPFS.
- There needs to be some discussion about how users can be sure they're using safe deployments of the Beanstalk UI and not a malicious forked version.
- The UI accesses most of the core data directly from the blockchain. It uses the sub-graph in some cases to make it cheaper to access certain sets of data where it is too complex to parse and process it in the UI. But for things like Silo Deposits, it pulls it straight from the chain.
- Designing the UI to access the data directly wherever possible means that all you need is an Ethereum node and a way to run the UI and you can use it. It is much slower to load data this way and can't be used for something like the Pod Marketplace, even for a single user, much less all orders and listing that have existed. But it is possible to use many of the core functions.
- There are a couple other options that are in the early development stages and aren't yet complete and tested, such as Substreams or Axiom (zero-knowledge protocol).
- Using Substreams is unattractive because it will require a developer that can work with Rust and they also seem to have little interest in supporting additional chains. It's going to be a longer and more involved process to use this and likely require consultation with their team. This is unlikely to be the solution we are looking for.
- It is very unclear what he capabilities of Axiom will be, although the zero-knowledge proof technology itself is promising. It will take major investment just to understand the technology and even start building on it.
Custom Indexing Solution
- A custom solution would be able to parse data from different blockchains and index them sequentially. It may store the data in a postgres or other database. It would implement custom logic to handle various mappings to track orders and things like that.
- Using a custom solution would be the most flexible, allowing us to start in whatever capacity is desired with a goal of making it more lightweight over time. This would require some research from somebody familiar with this technology doing a deep dive to determine how to build it efficiently. However, it would be easy to spin up a very basic version and development could prioritize what is needed by the products being built.
- Ultimately, whatever is built should be in the context of decentralized verifiable compute. It might be possible to start with a pared down version of a local custom solution to see if it could be built with Axiom.
- It is probably worth doing a deeper dive on the Axiom code base and see how close they are to being ready.