Illia Polosukhin: Near Protocol – From AI to High-Throughput Blockchain - podcast episode cover

Illia Polosukhin: Near Protocol – From AI to High-Throughput Blockchain

Jan 06, 20241 hr 29 minEp. 529
--:--
--:--
Download Metacast podcast app
Listen to this episode in Metacast mobile app
Don't just listen to podcasts. Learn from them with transcripts, summaries, and chapters for every episode. Skim, search, and bookmark insights. Learn more

Episode description

What began as an AI company trying to seek solutions in order to pay remote (unbanked) workers, Near AI became, in 2018, Near Protocol. Its sharded design was inspired by modern database architecture and large language model (LLM) training. Near Protocol aimed to solve the scalability trilemma, through a modular approach, combining data availability sharding with stateless validation. By abstracting away archaic blockchain standards, Near basically enabled decentralised full stack development and, in terms of UX, a distributed custodial solution via chain abstraction and account aggregation.

We were joined by Illia Poloshukhin, co-founder of Near Protocol, to discuss Near’s journey, from AI company to high-throughput L1 blockchain, and how LLM training influenced the modular design choice.

Topics covered in this episode:

  • Illia’s background in AI & ML
  • Scaling large language models (LLMs) and the role of attention
  • Stochastic Parrot vs. Understanding spectrum
  • From Near AI to Near Protocol and the role of LLMs
  • How Near abstracted the blockchain away and enabled decentralised full stack development
  • Defining ecosystem standards to improve UX
  • Chain abstraction, account aggregation and interoperability
  • Chain threshold signature
  • Near’s intent layer
  • Near’s modularity, Nightshade sharding & stateless validation
  • EigenLayer integration

Episode links:

Sponsors:

  • Gnosis: Gnosis builds decentralized infrastructure for the Ethereum ecosystem, since 2015. This year marks the launch of Gnosis Pay— the world's first Decentralized Payment Network. Get started today at - gnosis.io
  • Chorus One: Chorus One is one of the largest node operators worldwide, supporting more than 100,000 delegators, across 45 networks. The recently launched OPUS allows staking up to 8,000 ETH in a single transaction. Enjoy the highest yields and institutional grade security at - chorus.one

This episode is hosted by Meher Roy & Felix Lutsch. Show notes and listening options: epicenter.tv/529

Transcript

This episode is brought to you by Gnosis. Gnosis builds decentralized infrastructure for the Ethereum ecosystem with a rich history dating back to 2015 and products like Safe Cow Swap or Gnosis Chain. Gnosis combines needs driven development with deep technical expertise. This year marks the launch of Gnosis Pay, the world's first decentralized payment network. With a Gnosis card, you can spend self custody crypto at any Visa accepting merchant around the world.

If you're an individual looking to live more on chain or business looking to white label the stack, visit gnosispay.com. There are lots of ways you can join the Gnosis journey. Drop in the Gnosis Dow Governance form, become a Gnosis validator with a single GNO token and low cost hardware, or deploy your product on the EVM compatible and highly

decentralized Gnosis chain. Get started today at gnosis dot IO. Cos One is one of the biggest node operators globally and help you stake your tokens on 45 plus networks like Ethereum, Cosmos, The last year. In dydx, more than 100,000 delegators stake with Chorus One, including institutions like Bit Go and Ledger. Sticking with Chorus 1 not only gets you the highest years but also the most robust security practices and infrastructure that are usually exclusive for institutions.

You can stake directly to Chorus One's public note from your wallet, set up a white table note, or use the recently launched product Opus to stake up to 8000 ETH in a single transaction. You can even offer high yield staking to your own customers using their API. Your assets always remain in your custody, so you can have complete Peace of Mind. Start staking today at Corus .1. Welcome to Epicentre, the show which talks about the technologies, projects, and people driving decentralization

and the blockchain revolution. I'm Felix and I'm here with me here today We're speaking with Iliad, who is the Co founder of Nir and CEO of the Nir Foundation. Nir is a sharded layer one blockchain. So welcome, welcome Iliad, welcome back on Epicenter. It's great to have you for a second time. Yeah. Thanks for having me and Congrats on, yeah, 10 years, epic, epic achievement in this space. Yeah, thanks so much. Yeah, like you said, it's it's basically 70 years in crypto.

So we've we've all aged a bit yesterday in the episode we recorded yesterday, the 10 years episode, they had like the slideshow and you could see the progression of Maher, Brian and Sebastian like from their youth to their 40s or late 30s. So yeah, that's great, cool. Yeah, we actually wanted to start unconventionally with your background, but in in your case, it's a very interesting background in AI and machine learning. So we wanted to 1st sort of talk

about your work there. You're one of the authors of the original Transformers paper. Can you. Yeah, maybe start by like telling us about your start in the AI and the mouse space. For sure, yeah. I mean, so I started tinkering with AII think. Even in high school I was actually excited about neural networks as a concept and I worked for machine learning company that was pretty old school machine learning company starting from first year of college.

But when I saw kind of deep learning resurfacing in 2012 thirteen there was this kind of Seminole work at a time which now now feels like duh. But back then was very exciting which was they trained in neural network to encode and like encode the image and then decode it back into the same image. So pre training what we know now as and that model without any supervision learn to detect cats.

And so there was a neuron in the network which if you activate it it would it would generate a cat and like different types of cats And so it learns something like semantic without any training like in any like input data from here it's right just by looking at images. And so when I saw that and that was done by Google by Jeff Dean and Andrew Ng and they did it on a bunch of GPUs and like they they managed to scale it up.

And I'm like I want to do that. I think that's that's the thing that's that's going to, you know actually change things. And so I joined Google research. My believe all this was that natural language not images going to be the driver for reasoning and for kind of like intelligence because you know there's many, many species in the world like hundreds of thousands of species that see and only one species that talks and has language. Right. So there's way more semantic information language.

And so my team worked across a variety of things specifically question answering. So when you like type questions on.google.com we were actually running a neural networks to try to read our like pages that you see and respond to you was like a short answer so like you would see some damn short answers. Now the challenge was the neural networks at a time specifically recurrent neural networks were too slow to be put in

production. And so we were just using bag of words models, which means you just literally throw all the words without any order into the model and it kind of tries to figure out what's going on. And it worked. It worked reasonably well. But and this is where kind of the Transformers gave birth, was like we could not use RNN in any practical use case. And so we were looking kind of

for something. And so Jacob who was the manager and had like another team came up with this idea like they were using attention on top of words without any recurrence for for another task. And so kind of merging that idea was recurrence. Like can we use attention to somehow figure out which words are relevant in the order when you do answer questions or translate something.

And that kind of gave birth to the Transformers really was like we need something that's really performant, that can be highly paralyzed. And attention is really good mechanism, you know, logically to do this. But if you package it all kind of the way this models really work is that everything happens in parallel. Like the way I like to describe it, there's this movie Arrival where aliens talk in the whole sentence at the same time.

Like there's like a circle of scrub list, but they produce it at the same time. And that's kind of how Transformers actually read articles. It's not like one word at a time. It's literally reads the whole article, all the words in parallel and then like has multiple steps to kind of process it and reconcile this like the understanding of it and then it answers the question and so. So that lays out really well for the modern hardware GPUs that we

use. And so it allows to have like this massive kind of performance improvement which means also you can scale out the models. And so I've was a, you know, worked on that was a team of amazing researchers which now all went to do really cool stuff. And then at the time I decided to leave Google to start an AI company near AI, which was supposed to be pretty much

teaching machines to code. So my belief, and I still believe this, that now given this, this steps of models, you can change how we interact with computing. You can actually talk to computers and they do work for you instead of needing to have an engineer to write code for you, right. Which again like now it seems more obvious that that's possible.

Back in 2017 there was like huh And so so we started near AI but we only we gave us us a year because obviously at that time it wasn't moon shot and we didn't have that much resources. So we're doing some interesting stuff around data collection and some machine learning. But one thing we ended up doing is getting a lot of people around the world actually doing like writing some code for us, writing some descriptions for

the code. And so we we had to struggle to pay them because they were mostly students in China and Ukraine and Russia in kind of some other countries. And like some of them don't have bank accounts, some like Ukraine, for example, PayPal doesn't work. In China, PayPal doesn't work and so there's like no good way to do it like programmatically to send people money. And so we started looking at blockchain as like, hey, can we just send people money easily in in code.

And the answer was in 2018 the answer was actually no because even back then the fees on Bitcoin and stadium were way too high And and then as you probably know when you start on the blockchain rabbit hole, you cannot stop and just keep digging and you're like wait what is this? And so, so we kind of as we kept digging and researching different block chains and different technologies, we're like wait, we actually know how to build something of this sort,

right. So my Co founder Alex, he was building Sharda database company before and we have like you know systems background, we're like we can probably do this, but we can focus on user experience, developer experience while kind of solving the scalability underneath and making sure fees are staying stable. And so that's kind of how we went from near AI to becoming near protocol in 2018 and and starting this journey. So Elia in this current wave of

LLMS. Of course like this attention mechanism is a key part but another key part is just the idea of you know like just the idea of scale. I'd like collected a lot of data from the Internet from books and then pre train the model and of course the ideas of RLHF and all they came later. But the fundamental idea is you throw in a lot of data, you pre train, you make a big model sort

of produce good results. Did you anticipate that scale was going to work this well, and if so, why would did you use that approach in Near? AI, no, I I that that's a part that definitely kind of was interesting to see that as people scaled up the models, they became like they started exhibiting kind of properties like more and more sophisticated reasoning properties. And it's like it makes sense now that you know you think about it like the capacity of the model

is higher. It's able to like generalize better. It's able to kind of learn quote UN quote programs that it can execute. But yeah at the time that wasn't like particularly clear that like it will be that kind of step function change.

And so we yeah we were not at nearly I we're not doing that partially because we also just didn't have you know like we raised you know small C like pre C drown actually And we we thought we could get better supervised data instead and we did we did some pre training on like GitHub and and things like that. But we we didn't thought, didn't think of like training the whole Internet, that's like large scale and we did have resources to do something like that

either. And the other interesting thing is kind of like this. This attention mechanism also seems to like it's built for natural language processing, but also seems to kind of work across different modalities like such as like images and maybe video in the future. And like how does that come across to you right? Like is that unexpected or is that is that something you

expected in the past? I mean like when the Transformers were just in development there was like like the teams actually tried them on different modalities, I mean not like multi model models but different modalities and it was pretty interesting to see it worked really well. So I think it's that was kind of known that it works on on different modalities pretty early on.

I think the kind of the intuition there is really that you know the way kind of we work as well as very much like like our our eyes actually like move all the time every like I I forgot how many milliseconds. And so we actually kind of pay attention to different parts and then our brain kind of reconstructs the image right at different levels and kind of you know residential language same right.

You read sentences you like build some semantic meaning and then you know it kind of continue building out this the meaning of the what you read. But sometimes you like zoom in on specific words when you need to answer a question and so like I think like generally speaking there is like intuition behind this but obviously again it's like it it's interesting to see how well it all works right Definitely you know like we had we had a pretty good models like even before it.

It just like they were super slow and like non you couldn't use them in production at all but this you know obviously like the the scale is which for example Open AI went and and scale it up and and by the way they did a tremendous amount of work to make it work like it's not we cannot take it for

granted. They're just like oh we just increased parameters and hit enter like no it was a ton of work across the board from you know low level engineering to like fine tuning to you know they changed some of the model kind of details of model architecture as well. But yeah, like the. It's it is. It is It was surprising for me like, I think like that when it went from 2:00 to 3:00 that was like interesting like two. It was kind of like, OK, yeah, I get it.

Like we've trained models like that at Google kind of thing from 2:00 to 3:00. It was like, OK, that's that's really interesting because I can see the, you know, there's like something more now happens and obviously it's 3.5 is where like, OK, yeah, that's like it actually learns something that is like beyond just language modeling, right? Like there's some reasoning that this is extractable now through kind of this instruction fine tuning.

On a high level, I'm I'm actually curious what your stance is on this stochastic parrot versus understanding spectrum. So, so there are, there are people in the AI community that that say that LLMS actually don't understand anything. They are stochastic parrots in the sense that they have understood the statistics of what word follows what other word in language because they

have seen billions of examples. And when you talk to an LLM and it's generating words, it's just replicating the statistics of what it has seen in the past without actual any understanding behind behind the box. That's the like, at the extreme, that's the stochastic parrot view.

And on the other extreme, perhaps there's a view, maybe like the ES upscale view, which is kind of when you force a model to predict the next word and you force it to do it again and again in order to predict the next word. It has to Start learning something about the world itself to do the the job of prediction well.

And in kind of trying to predict it well, it is forced to learn about the world and so it has actual actual intelligence about what world it has in it is in. So it's not just a stochastic parrot. This is actually when you're talking to GPD 4, you're talking to something which has understanding distilled into it and they seem to be like these two extremes in the in the space and I'm I'm curious like where you stand on on that on that debate.

Yeah, I I mean I definitely closer to Discover's view like from my perspective kind of you know it at the end is bunch of math, right. And so like you can kind of decompose what this math is doing and you know try to build an intuition around like types of transformations. It can, it can or cannot do. And so from my perspective kind of you know the first step is you take the document and you

embed it, right. So you went from words into a multi into dots and multi dimensional space, right. So I mean let's let's for a second imagine it's two-dimensional although it's multiple. And so there is like the words that are similar, right. Are you know close to into space. The words that are far. Now you have a next layer which transforms us words, right, to kind of give them more context.

And so just, you know, think of it as rotation in the space and then you have a tension which is you know, you're trying to kind of given the, the current word, you know, try to pull in the context of the words around it to give to give it more semantic meaning. And so that's another transformation, right. So like in a way you take kind of set of words, right, and then you kind of keep transforming

them. And so it it what it learns is the transformation function which in a way is a program. It's a program that is trying to transform the words into a level which is useful then to predict next word right or and then like later respond to questions. And so kind of is this like a pure stochastic parrot where it's like well pure stochastic parrot we had when we were doing just like you know, we were generating Wikipedia articles for example, right.

You just give it a name and just say generate a Wikipedia article like that's pure like you know there there's it just makes up stuff because like that that name doesn't exist right. There's no there's nothing So just generate something that looks like an article But when we when we starting to look at like OK well how would you answer to this question right. It it to be able to do that right it needs to kind of

process information right. It it does this kind of transformations on the on the article and like it's trying to connect contextualize that and and give give the answer. So in a way like I think of it as it's learn some set of programs that like our world has right. So like it's not a complete word model, right.

It clearly has a lot of gaps, but it is a kind of set of programs that our like world model has that it can apply to be able to to answer well or predict next word for a training and that in itself is really useful, right, as we see. But it's also because it has so many gaps it it, it has issues with doing some you know, kind of specific things and the more precise it needs to be, the less well it does, right.

Because it kind of ends up being like if either as a program, the programs are very probabilistic and kind of semantic versus you know, if you ask it to like describe the steps of of something. But at the same time a lot of the things we do is kind of like there's like few thing core things and then everything else you kind of fill in

automatically, right. So that's why it's really good like even at coding like most of the most of the coding we do right is actually kind of boilerplatey and so and so there's like few not just you can actually get to like a reasonable code and that's why I think like things like copilots are in pretty good products in result. Cool. So look, turning to applicative views, so now this LMS are are pretty amazing and you have some applicative ideas on applying them to the near ecosystem.

So yeah, what? What are they and how do you see that unfold? Yeah, I think I think of this kind of across 3 dimensions. So the first dimension is actually less about AI itself and more about our kind of society. And this is the idea that kind of as more content is generated as there's more kind of information wars in general misinformation. And again, the important part to note, misinformation is not AI

problem, it's a human problem. the IT, you know, we are in crypto space and so Byzantine generals is something that our space is based on. And that's literally the, you know, the miss cite citeable misinformation like.

And so the idea of like misinformation of of malicious like attack on information is something that exists from, you know from like early on. And so from my perspective, the way to kind of start solving that is to to bring the kind of security, cryptography and reputation to a level of of the content of the of individual pieces of content.

So right now, for example, we are, you know, using websites, we have HTTPS and so we have, we have some set of security guarantees around accessing specific websites. But the content on the website can be coming from anywhere. It can be saying anything and there's no way to kind of maintain reputation, contacts, comments, etcetera around it.

So we need a new set of standards around that, so that you can hover an image and it shows you or a video or piece of text and it tells you like who published it, when it was done, if there's any side comments or contacts etcetera from reputable sources that should be attached to it. So for that we need blockchain, we need, you know, set of standards, we need browser support and we need kind of publishers to be supporting this.

And I think that's a really important part for our society generally because otherwise we're going to be living in a world of kind of, you know, all the content is like you never know what if it's true or not, right. And it's constantly like kind of manipulation around that. Now kind of the second pillar for me is I call it kind of

decentralized AGI, right. So if we assume you know this models are getting more powerful, more intelligent, what you definitely don't want is a single company or you know two or three companies deciding what's right and wrong for this models to do. You don't want to like them to decide what you're allowed to do and what you're not allowed to

do as models. It's also like it's the same thing that happened with social networks, like being a kind of moral police for the world just doesn't work. The world is very multi dimensional. Something that's legal in Amsterdam is completely legal in a lot of other countries and the other way around. And so like you know what moral is, is even more complicated.

And so it's really important to have community be governing kind of the alignment safety as well as kind of the, you know, instruction data sets that these models are trained on and as well as like being able to validate that the model you run is actually the model that you wanted to run, right. So right now if you call, you know, GB TAVI or Google API, you get a response, you have no idea which that could produce that

response. You have no guarantees that it was the model that you wanted to run. And actually sometimes it's not because they're trying to optimize costs. And so like how do you actually have this guarantees? And especially for something that's mission critical, right, like like if I'm doing trading on this, if I'm doing healthcare, like any kind of business decisions, right. You want to make sure to, you know you're accessing the model that you have predictable

parameters and outputs. And so for that we need decentralized inference. We need kind of model marketplaces, We need kind of community data, crowdsourcing and data management governance and took kind of the whole stack of tooling that really manages this. And then, you know on top of this you'll be able to to kind of interact with it in a hopeful like. I think the other way is like making sure it's privacy preserving so that when you interact with it, you have it.

So there's a lot of work to be done. There's a lot of like there's a bunch of startups doing decentralized inference. There is still privacy gap. I think that people are researching, but it's still pretty far. There's some data marketplaces, there's some other kind of pieces, but it's not really, I would say like combine into like a product story yet. But I think like that's a really important for like humanity

period. Because otherwise, you know, like tomorrow you go to your favorite, you know, AI model and it says like, oh, you're banned or you know, you use the incorrect word and so now or something, right. So all the usual stuff we've seen before. And then finally, I actually think the the flip side of this is local models, right? Because although like this big models, they have the world knowledge, they have maybe access to lots and lots of context.

But actually what you want most of the time is a model that knows everything about you. But you don't want all this data to go anywhere else, right? You want it to live with you on your machine, on your, you know, your private encrypted data store and you want to model the table to access that. So you want a local model that is personalized for you. You control it. It's not affected and manipulated in any way by, you know, advertisement giants.

And so it's actually on your side and there's just responding kind of the way you would like to not the way you know tied once you or whatever. And so I think that is a really important side of kind of as well.

And so we're actually been playing around with like Edge Intelligence and I did a couple events and being kind of talking with some projects around this space and it's like it's actually it's lasts VEP 3 like you know sense of blockchain, but it's more of a three in a sense of principles, right. It's user owned AI is you know controlling your own data.

It's like all of those values that we to talk about and I think that and kind of the web series of custody will be converging more kind of in a in in on the principal side, right maybe and like on technology side as well and this kind of the area I'm I'm most excited and working on right now. So in practice how how are you like approaching this? Are there like teams you are funding or is like there AI team in the year or how? How can we imagine this?

Yeah. So we've been working with some of AI teams. We actually just had a Neo con about a months ago and we headed AI track there with some projects presenting that we already working with as well as kind of I'm working as like advisor with a few projects kind of more closely and we do have I would say like AI efforts more on also just automating our own operations like. So the other side of this is I think kind of the ecosystem itself should become AI enabled and over time AI ran.

So like ideally my, you know, my job and kind of the job of coordinating the ecosystem should be done by AI. And by the way the AI is a kind of like this approach actually solved the core problem of humanity and of resource

coordination. The core problem of humanity is principal agent problem is that when we want somebody to to to do stuff on our behalf like we select, you know, in elections or we hire someone to manage our money or something else, they have their own needs and and they have their own wants and so their decisions are usually not fully aligned with us who hired them. So that's called principal agent problem. And so AI actually being the agent that behaves on our behalf, acts on our behalf is

the way to solve that. And if you scale it up to kind of governance level, right, like actually having AI, being the actor that you know, makes decisions based on what the population wants, is the way to solve a lot of the current challenges was, you know, when you're like someone, they do stupid things, right? Or not, things that they promised to do. Kind of that's a way to like really address it. And so there's a really interesting kind of future of governance there.

But like, we can start applying it now in this decentralized ecosystems because they're already fully digital, they already have kind of like all the actions are in chain, right? So you can have traceability, you can have like veto power, etcetera, if something goes wrong. And so I'm really excited about also that side of the applying AI in in the three space and obviously you need that whole AI like decentralized stack to do that.

But we are kind of starting to do it from bottom up on on our side just in foundation for example like hey what are things we can automate, what are things that we can like start leveraging this technology for as well as maybe build some of the tooling for developers to build tariff AI enabled things in in in the space. We also have, yeah, a bunch of projects that are kind of experimenting with this across different areas. Yeah that's super awesome.

I I also saw actually your Co founder like the Alex working directly on like smarter LLMS. Can you maybe also like what what's that about? Is that related to me or is it like some sort of totally different thing or what can you share about that? Yeah. So I mean it's it's a stealth project right now. So I'll I'll not go into too detail maybe you'll have him you know in a in at some point to go more in depth into it.

Well, yeah, I mean we kind of so I'm advisors there and we working kind of I would say side by side. But yeah, he's focusing more on the lower level and like kind of preparing for the future of of this as well. Yeah. I think, I guess maybe you you're mentioning right, like AI sort of also like making our life easier in the sense of operationally in the organizations, but also I guess, yeah, in the wider society. And I guess that's always been like a huge focus of Near.

So yeah, we wanted to sort of dive also in that side of Near where basically you're branded now in many places like as the blockchain operating system. And I think, yeah, one of the core features around that is like sort of the the UX focus of new. So maybe yeah, can you explain to us how Near has sort of approached yeah basically usability for for developers and and users in the in blockchain systems and and what you're currently doing there?

For sure. Yeah. So I mean this was our vision from the start because when we started ourselves kind of diving into the blockchain and again this 2018, so things were different. You know you needed to install Mist for the and so the I mean the experience was pretty like painful and and it's also it was built on top of kind of a very different set of primitives I would say like conceptual primitives said than what normally people both users and developers expect right.

So, you know, you need like to understand the Xerox wallets, you need a seat phrase, You need to like kind of pay gas. You need to like have, do all these things which are like strange when you, you know, when you're just starting.

And what we've tried to do from the start is like how do we design kind of still like a blockchain that is secure that it has all all the same properties that we all want, but is able to kind of hide a lot of this complexity, ideally most of it, and make you know blockchain kind of abstracted out such that developers when they build applications can just build like as close to normal that to experience. But using the benefits of that three, using the kind of all of

the value. And then also enabling users to have like more composibility, right, more ownership kind of being able to interact with multiple kind of applications and and have this like transportability of data. And so the near itself, right, kind of was designed was this. So we've like our accounts for example, you know the account obstruction part of the accounts have been designed from scrap from the start on the protocol level.

There's like a bunch of kind of differences that was done including that accounts themselves are you know just a username that follows kind of domain name structure. We have lots of different keys with different permissions which allow us to have like multiple devices securely. It allows to delegate access, it allows to like applicate the front end of application to have a session key for example to transact for a specific set of kind of interaction.

So kind of all of this functionality comes in by default. And then on the developer side, the choices we made around first of all choosing Web assembly which but at this point is like seems that everybody's kind of agrees on, but pretty much it's like it's an engine that runs in all the browsers. It's something that you know is like on billions of devices at this point it's supported by you know, large network of kind of developers. It runs on edge. It you know it supports lots of

languages. You can run a lot of software in it and so so we kind of picked that and made it really easy to build like in a way from a developer perspective what when you write nearest my contract it's really just a service which has message like messages in and out and you have a kind of local key value database which is pretty much like the limits there are so big that like I don't think anybody ever hits

them like you can't you. I think we have contract that have like 4 gigabytes of storage in their in their database right. So you can build like massive massive contracts specifically you can build all the chains as a smart contract on there. So we have Aurora which is an EVM as a smart contract just like taking you know the EVM that's usually run people a separate chain just put in smart contract their their database is where all the state of the of

this storage right. You can do the same as Bitcoin. I've I've been suggesting somebody to like fork Bitcoin and put it on there make it ultrasound money. We have JavaScript running as well. So you can run JavaScript smart contracts you can potentially do Python And other stuff. So like it's kind of enables developer experience across the board. And since then we kind of following the same principle is like, OK, well, now that you can build anything on smart contract

side, what's the next part? Well, actually you want to get the data out of this out of the blockchain and blockchain are not optimized for reading data. They are all kind of we've tend to optimize them for writing and kind of maintaining security. And so for reading data you want a completely different data structure and so that hence there's like this principle of indexing and kind of in the way of chain computation.

And so we've been building indexing framework and that actually culminated in what we call query API which is a service that indexes that you can like write a smart contract that describes the indexing of data that executes off chain. So in a way it's like a off chain computation framework that allows you to store output of that computation in kind of SQL databases that then you can query and finally, well OK now you have back end and and middleware, now you need a front

end, right. And again it seems weird that we are like oh you build everything decentralized but now run a server on a specific domain that you will need to maintain. It's like OK well that kind of violates the whole part point of what we're doing. So, so we created this kind of decentralized front end framework that allows to store the front end code itself on chain.

So again the smart contracts code on chain, the middleware code on chain, the front end code on chain and now anyone, any kind of we call them gateways can render this code on a user side, right? So we have a desktop app, we can have a mobile app and we have obviously web apps that can bode that from the blockchain directly into your browser and render it there. So there's no kind of middle server that's needed to render you, you don't need to have a

domain. You can obviously if you want to and so you can just you know launch kind of part of your web app as as this decentralized front end component and now it will live forever on blockchain, right? But side by side with your smart contracts, have the same operability, have security, cryptographic security, who has

it, have versioning. So if I as a user don't hate a new version I can go to version before and so like all the same properties we really like about smart contracts we now get for front ends. So, so all of that is really enables like a full stack decentralized development that is you know familiar with to normal developers. It's React JavaScript components, it's JavaScript for middleware indexing it's JavaScript Ross C++ and like other languages for smart contracts.

So you have like a full stack decentralization that you can have. And interestingly as we were building the front ends, we realized actually the front ends can work with any blockchain. And so we kind of just turned on all the Avms and some other blockchains and people started building all the AVM fronts as well. So we have a UNISWAP for example, for linear, the kind of official UNISWAP front end is served out of the decentralized

front end, right? Because by the way, it also doesn't charge extra fees and we have like partnerships with others, the KVM, Mantle, etcetera. And so the idea is like actually as you start looking from that lens, from a user lens, right. As a user, I don't really care which blockchain the apps is on, I just want to use them.

And like if you go to you know some like you know some of this gateways where you can access this front ends, you can just go and search for whatever app you want, click on it and start using it that that's how it should be. And so, and this is kind of where we get to this concept I started with which is like, hey, we want to abstract the blockchain for users and developers.

We're getting back to it with kind of this now that we have this full stack decentralization where like actually this works for all blockchains, for all chains, for roll UPS, for whatever, because you can actually abstract out all that on the front end side and make it really easy for people to interact with it. And so hence we kind of started going backwards now with some of the other launches we had, right, allowing pretty much as a kind of how do we make it really

easy now for one experience to to unite all of the block chains and kind of recall the chain abstraction principle. And so this goes into like GA and and some other things we, yeah, we can discuss. So, so India is it, is it correct to imagine so when you talk of like this indexes service or the service of hosting a front end, is it correct to imagine it as the indexing logic or the front end

logic is stored on the chain. But then there is some kind of off chain actor that is actually taking that, taking that logic and the data and actually serving it much as a traditional server. And somehow the chain is guaranteeing that this server's work is correct and it is compensated. Is it? Is it correct to imagine it like that? Yeah, pretty much. So the idea is, I mean similar to maybe blotching validator nodes as well, right?

There's a kind of a logic that is conceptual and then all the validators are doing that job and like you can always have, you know, more validators, less validators. It's kind of independent of that. Similarly, yes, the indexing logic and the front end kind of source code itself is stored on chain and so any server can run on and kind of create the same, you know outcome from this,

right? Again similar to RPCS for example, RPC server right is serving your data but it's you know anybody can run RPC server and get the same results. So like it's it's part of protocol. In a way it becomes part of protocol and so similar thing we're trying to do for about both front ends and middleware indexing as well. So maybe one way to think about this is that, so on on on the TDM, right?

If you look at the TDM, there's a basically a blockchain and then there are separate protocols like the ENS for naming your blockchain address to a human readable name. That is the graph which kind of like indexes a smart contract and kind of presents historical data about the transactions and events in the smart contract. And maybe maybe there are other

examples that I'm missing. So in the TDM layer, these are like different systems and usually they are competing systems as ENS, but they won't be a competitor to ENS the graph and they won't be a competitor to the graph. But in near NIA has kind of taken the philosophy that some of some of these things are like are like really key to the UX of a blockchain and therefore they should be supported out-of-the-box by the layer one itself. Is that is that the philosophy here?

To extend, yeah, I think, I think the way to think about it is it's more than just layer one, right. Like at the end when we are interacting with applications on any of these chains, like there's a whole host of tools and and more importantly standards that we are interacting with, right. And so like ERC 20 for example is a standard and it's a standard that kind of came out

of the application space. But it's now like you cannot imagine the CDO without the ERC 20 standard and So what we're doing here is really defining standards for this key primitives that are just going beyond just you know token transfers but going to like how to how to define indexing, how to define this interest run it's now implementation of those things can like you can have many implementations, you can have you know mobile render and web render.

You can have indexing like you can have you know external partners who are competing with each other, how to implement it? Same same for RPCS, right? RPC is a standard but then the way it's implemented right can be very different like underneath you may be like cached everything in database and maybe using cloud flare like whatever the architecture you

want to use. But the standard is there and I think what we've been trying to do is define a standard and I mean have a reference implementation but for for this more key pieces to make indeed the experience more aligned and kind of have this like singular journey for developers and users that you know is cohesive. And yeah, like the way you know some of the things like you can have businesses around the standards that are, you know, very profitable.

But like the core principle for me of decentralization is actually in the standard. It's the fact that like if you define a standard, it means that you can swap in and swap out any participant. And so you're not you don't have

this like lock in effect. You don't have the effect of you know you go to a bank and you cannot move your money out because it doesn't allow you to. Or you know the cannot cancel your telco provider because like or like your positive telco providers don't even work for you. The here we can always have like a competitor that comes in and if they're more effective and can provide better prices people can switch to it. But the stand because the

standards is the same. And so for me that's kind of the key principle of like Web 3IN general. And so I think the challenge that I've seen is like not having the standards actually leads to kind of huge fragmentation of experiences. And as well, like actually Monopoly has been built because like now that you've built all your software towards some API, you cannot switch because nobody else provides us. And like you need to rewrite

half your code to do that. So how much is kind of analogy with the Apple ecosystem versus the Microsoft ecosystem for desktop, how much, how well does it map? So in a sense when you look at kind of like the the Apple, the Apple ecosystem, it's a company that has kind of maintained control over, it's kind of like operating system, supply chain, it's way of like delivering music, it's way of delivering books, it's way of how kind of applications kind of like appear

to the end user. And in the in the beginning, I think they also wanted control over the hardware, but maybe they have retracted on that now. Whereas like kind of like Microsoft is 1 where just like the raw operating system and then applications emerge. And if there are standards needed for their interoperability, the market

kind of like figures it out. And from the outside it feels like OK. Near is kind of like more going towards that Apple philosophy that we are going to define all of the standards for, for many of the things that are key determinants of the user experience. Whereas other ecosystems like Cosmos or Ethereum might be more kind of like the Microsoft approach where we are providing like transaction throughput as the center, the account model as

the center. And then kind of a lot of the interoperability between the standards is left to the market to figure out how much does that analogy map and how much does doesn't map. I mean the, I would say the. Part that's like agree on is we're definitely trying to focus

on user experience, right. And so with that, it's important to figure out like what are the touch points that you want to have standards on. Again like my perspective is for example RPC, Jason. RPC is part of Ethedium standard, like it's part of kind of the protocol even though it's actually not but like it it for both accounts.

If you try to change it RPCAPI like you will break everyone And so, so we kind of see in a similar way, right, like if RPC is part of the standard, why not some other parts. But as I said like you know Near for example has like number of contributors that they're building things like the, the actually the VM that's built right now for the decentralized finance is built by proximity, right.

And for example you know the query API kind of other companies can implement the same standard and provide kind of better services. So I think the idea here is that by defining the standard we kind of actually opening up the market for people to fill in like with better products in this. And again like it's pretty early

still. So like a lot of stuff we still build like reference implementations, but similar as a CDM by defining a standard for protocol, it opened up a place for all of these clients to be implemented, right.

Like that's kind of the idea is like you you, you define a standard and then you open it up so that others can contribute to it. In the same way versus competing on AP is and competing on like kind of you know in a way marketing what's happening right now in in in like talking price, what's happening in the CDM for some of this like infrastructure

tooling, right. It's like can can we get a bigger AirDrop by using a product versus like hey, this is a standard, everybody will be using this standard and so now what's the best product people can build for the standard. So I think like that's kind of the difference. I don't think it's as applicable to like this, you know big commercial for profit companies versus like this is an ecosystem that we're building and really defining more kind of this I would say like layers of the stack.

Going back a bit to like this, what you said about the switching cost from your telephobic, I guess like related here. I guess one big thing in blockchain is generally like bridging or like if you want to switch the ecosystem you have to like go to some other chain and then like move the liquidity there which can be like cumbersome. And you did mention the chain abstraction for a second there and I I saw on your Twitter a bunch also like this concept you have like of account aggregation

that you teased. So maybe you can yeah, can you talk, tell us a bit about like what what are you doing there or how are you like sort of solving this interoperability problem in in the blockchain space? For sure, yeah. So that that's a very important topic. So although we have started building bridges I think so the our Rainbow Bridge been built from 2019.

So I think we started building you know kind of in line with IBC kind of timing and we've been, we've been running since I guess in the beginning of 21. And at the same time like bridges are really bad as a concept because they created

honeypot for security. They are the place to siphon off assets and like there's you know if there's any attack on the protocol itself bridges kind of how you exit and and they like that just the amount of failing modes between different block chains is pretty like it's pretty big right.

But between multiple block chains it's like insane like you know chain stop blocks didn't publish like all those things you need to like as a developer now you need to handle and then on application side, OK yes you know the fungible tokens transfers is maybe reasonable but as soon as you add any logic, right be that you know rebasing or be it LFT. Now when you bridge it you lose all of the logic on the other side and so so the concept I've been kind of exploring for a

while now. I was calling it originally remote accounts, but we we kind of reframe it as account aggregation. This idea that ideally you're going to have one account and there's mapped accounts to this on other chase. So imagine you know you have my root dot near account on near and then I have an address on this idiom, I have an address on Bitcoin. I have an address on Solana which I control with this

account. And so now if I want to buy a Solana NFT, right, right now I would need to like set up a new wallet, you know bridge some stuff to Solana by the NFT. And then I like, I don't know and then like go and look at it, you know from time to time because I'm mostly sitting on here or you can you have the Solana address that's linked to your to your new account. You pretty much through this by an NFT for this address and we can talk how that works.

And now you have the front end that actually shows you everything you own across all of this chains from all of this addresses. And the way mentally to think about it is when you go to Binance or Coinbase and you sign up with your Binance or Coinbase accounts, you have addresses on all chains, right. And I mean they they are deposit addresses usually. But imagine those addresses were

actually normal addresses. You can use apps and buy NFTS and tokens etcetera Wiz so that and but your account is your you know, Coinbase account. And so that's kind of where you're, you know, like ownership is.

And so that's what we're trying like we building, we're going to be launching end of the first quarter is this concept of account aggregation that now allows together with decentralized front ends allows to actually collapse the kind of this whole, you know, multiple chains switching networks, you know, bridging all of this into a very simple experience of you get an account, you know you deposit some funds into it and now you can transact across all block chains, across all of

their apps. And it kind of will get executed on on your behalf on those chains. And you have this addresses but it's all self custodial and all kind of hidden from you. You don't need to think about gas fees on those chains etcetera. So that's kind of the experience we're going after. And again, this is just an extension of what we've been building was near by trying to abstract out the near blockchain, we're just like OK, well we can actually do that.

It's the same thing for everyone and really provide like a unique and valuable experience because you know anything multi chain you want to build near will be actually the place to build it because you will be able to transact across all of the chains without having to bridge, without having this like

complexity. You want to build for example Bitcoin D5 well on near, you know every near account or smart contract will have a Bitcoin address you can deposit to it can start you know doing stuff right. And so that's kind of conceptually what we really bringing to market with this and like kind of finishing our I would say arc of chain abstraction that we started with doing near in the first place. So on a on a high level India, I'd like this idea is is in the Cosmos ecosystem.

There's a chain called Neutron and because the Cosmos ecosystem has IBC, so Cosmos chains can bridge to each other in quite a good way. Neutron has the idea that in Cosmos you have the idea of delegated account control, which is like on one chain you have an address and that can control many other Puppet addresses on other chains and neutrons.

Trying to build that kind of that that Puppet master chain where like you would have your central account and you will control other addresses on a lot of chains over IBC through Neutron. Does it feel similar? But the reason it works in the Cosmos ecosystem is you assume my BC that there's a secure bridging solution underneath

available. For this to work in Neutron my I almost start to saying that OK, the only way like this can work for Near and Solana. For example, having like a address on Near that in control of puppet address on Solana, you need a secure bridge between Near and Solana, is it not so the bridging problem? Solving the bridging problem seems like a prerequisite to this. Yeah, so we're trying to go away from bridging almost completely.

I mean there will be some places where you still need bridges, but so let let's look at Bitcoin as just to get way more clean. Example, right. With Bitcoin you cannot have a smart contract bridge because, well, Bitcoin still have smart contracts and so the only thing you can do is to own addresses. And so the core idea here and it's conceptually the same as yeah, what Neutron is doing, but the core idea is different.

The core idea is that we make near network itself be able to sign transactions for other block chains And so near network becomes in the way custodian of all of this address mapped addresses on other chains. And you as a near user telling network right be that through smart contract or or or user user interaction to sign a transaction on Bitcoin to send some bitcoins from your remote address from your delegated address to some other address right.

And so because of this like you don't need to actually bridge Bitcoin to near to do anything right? You you just literally the bitcoins live on Bitcoin network. The you know OP coins live on optimism the you know Salon and NF TS will live on Salon, NFT on Solana and I just control all that by just sending transactions there. But as a user like I just interact with near and I kind of pay near guest fee which is very

small. I say like do this, you know I attach whatever also you know if I need to buy something etcetera on near and then we have kind of intent relayers that actually execute stuff like you know the transaction gets signed by near network and then intense relay of you know sends that transaction on your behalf on the other. And so there's no actual like bridging.

There's no kind of security kind of issue where like if this bridge gets broken or whatever or that network gets forked etcetera, like none of that exists. And because near account there's also like a very interesting and kind of a little bit crazy thing because near accounts are are actually tradable. So you can actually list near account as NFT and somebody can buy it and get access to it because you can rotate keys on there.

What is allowed to do is you can have lots of assets across all kinds of networks and then you can list that as a bundle on near as like you want to sell some BRC 20s, some so on NFTS, some Ethenium NFTS and some and OP coins and GMX at the same time. You can list all that as a bundle under one near account and then somebody can buy all that with one transaction on near paying near transaction fee and within one second block time.

So you don't need to wait for Bitcoin transfer, You don't need to wait for all of this. You can do it on what so you can actually start. Bundling all of these things and trading kind of across all chains on near very easily without actually sending transactions of bridging anything anywhere else. And that's kind of the the shift that we're trying to do. I called on bridging that we

like. You have the account level kind of ownership that's maintained indeed, but it's maintained by very specific security parameters that are near parameters. And then if if the, let's say Solana network fails for whatever reason, there's no bridge problems, right, That would you know, rise from from this like because you own stuff on on Solana. So whatever Solana has to deal with, right, like whenever it recovers etcetera, like you will get it back.

But like it's it's kind of the same like you know you have this kind of relationship with that network but not like there's no bridge that you need to deal with and kind of think of as like an intermediate you know complexity. So that's the idea. It's like you know again we're going to be rolling out more documentation.

It's like we have a test net version coming out for people to hack on in in kind of January. And so we we actually invite people to start building because again like a multi chain experiences like you'll be able to build this this way easier because you don't need to think about all of the complexity of like oh this message didn't deliver that that work is like paused. You know like something crashed because of inscriptions like you don't need to deal with any of

that right. It's like you can literally sell your your you know network failed. You can sell the account that that has assets in failed network right to somebody else for example if they want to take that risk, right. So like you can do that without having that network life Even so. So that's kind of the the, the level of experience we won by and like this leads to fully obstructing the block chains

right. Because now from a user interface I just go I use the app and like I just see that I'm using my for example near account and it doesn't really matter for me that that was a Solana like NFT that I bought right. I just see it in My Portfolio view and like you know for that we need like indexing of Solana and all the other chains data. So the same stack there we need decentralized front end that kind of aggregate all this and

so kind of that. That's like how we package all the stack into by abstracting the blockchain. So quick quick nerd question which is so OK so Nia's like this is awesome. First of all I mean NIA becoming like a distributed custodian essentially imagine it as like Coinbase but distributed and the distributed custodian can have hot wallets basically on all of the other all of the other

chains. But as an engineer my my my questions really starts to be in in Bitcoin you have like a single single account or a multi sig account. That's what Bitcoin provides available right? Like it, it assumes that there is like maybe like 1 private key and then one public key and there's a signature to that public key. Whereas near as a distributed network has lots of validators. So how do what fancy cryptography makes this makes this work? Yeah, so it's called chain signature.

And so this is a threshold signature with where the valid as validate is rotated. You can maintain actually the same set of public keys. So even though you rotate and like have different parts of the private key being rotated, they all like when they sign threshold signature you get the same public key then. I mean you can have derivations of this, so you can have like as many public is possible, but they all deterministic within the whole blockchain, so, so

that's a pretty cool technology. And yeah, like it's kind of reasonably new. Some of the folks from the city have been pioneering that and yeah, we've kind of leveraging that as a way to have needed to become this, this decentralized kind of custodial. Right. I think maybe also XLR works a little bit like that or am I? I think, But anyway, one question that I had like in this scenario where you have the divider NFT on Solana, you need the liquidity right on Solana as a user.

So maybe I have like funds on the air, but I don't have on Solana. Like is there some system that you're thinking of to to balance that out without bridging or

yeah. Exactly. Yeah. So we so this is where we call them intent through layers or I mean we'll we're still shopping the name but so this is the idea that on near we have this well we have this principle of trial account to this idea where I can send you right now a link you click on it and you'll have some near in it. So you can do stuff on near but

you cannot withdraw that near. So we actually like, we kind of like what what it'll do is like actually send you a one time use private key which when you click it actually create a new private key in your in on your browser switch that private key. But that private key is limited access to that account.

So you can transact but you cannot withdraw funds And so that kind of concept applied now to other kind of chains in a way a lot what it allows to do is we can have other parties to, to fund the account to execute things, right.

They can put some Solana tokens to pay for gas or for NFTS but you cannot withdraw that by sending a like direct transaction to withdraw Solana. So, So what this allows now to do is you can pay the somebody on near with near token and then they will put Solana tokens there and then execute your transaction. And kind of by doing that right, we kind of have pretty much a way to that's why I say it's

intent, right. You say like my intent is to buy some Solana thing but I don't have Solana token like here's punch and near tokens execute that there and so and you know now you need somebody who has liquidity on all the chains to execute the stuff. But that's like having a third parties doing that is way easier right than to have like whole bridging and automatic execution. So this is like, yeah, I really like some sort of fee attached to it and the relayer can grab it.

It's not like a blockchain network or anything. Yeah, yeah, exactly. Yeah. There's like a surf body, like, you know. Like a market maker or whatever. Whoever it is, it's. Like yeah, any market maker or any like bot arbitrage bots can do this kind of stuff pretty much and they also is doing that they'll just relay the transaction as well.

So like you don't need to actually also send the like submit transaction because like the valid is only signed transaction right now and somebody needs to like actually ship it to peer-to-peer network. So they they will do that as well, yeah. That's pretty awesome. Yeah. I like looking forward to reading more about it once the the more documentation is there and stuff. But yeah, thanks for for sharing

it here. And yeah, I guess further in the near journey like we we didn't actually talk about much about the chain itself, right. I think you were basically one of the first, if not the first like sort of sharded block chains and and been like staying with that sort of narrative wire. I think the others have pivoted from that. So yeah, can you tell us a bit like how has the like near sharding developed or what?

What is sharding actually again for people that forgot about it and you know where, where, where is it going? Yeah. So as I mentioned, right, my Co founder Alex who was you know, building sharded database, I mean I'm coming from Google where everything is charted just like you cannot have, you know, billion users and put them into one database. This just doesn't work. And so and like on my computer and so for us it was like, you know, kind of pretty obvious that you need charting.

And so sharding, I mean at at the core of the idea is like you as you process, you know, as you store more data, as you process more transactions, you need multiple machines doing work in parallel and you want this machines to be kind of doing similar work, right. And like distributing load And ideally as more load comes in, you actually increase number of computers, right. So this is how all of the VEP 2

giants work. You know, again, imagine your Gmail, right, or imagine Facebook, right. There's like a database underneath which you know is charted. It has hundreds or thousands of servers that store for example user data. And you know when you're a user requesting it routes you to the server where your user data is and retrieves it and then when you need to update something or process you know transaction, it kind of routes a transaction there. So that's kind of the core concept.

And like you know again logically you cannot have like you cannot have billions of users using the same like 1 server, right? And this is what's currently happening where for non sharded systems it means like they relying on pretty much one server replicated, but one server nevertheless to process everything that happens on their chain and so. So for us it was kind of, you know, pretty obvious when to do this.

Now blockchain adds extra complexity compared to that too, where you have all of the, you know, security that you need to deal with. And so we've been kind of obviously iterating on, on a design kind of within this

conceptual thing. And so we introduced Nightshade back in 2019, which was our sharding design where in a way every single near contract or account is actually a separate chain and we just bundle them in such a way such that as users and developers you don't know about it, right. And so we kind of bundle them to the number of machines that you know parallel processing machines at a time you need to.

And so again this is very similar how VEP 2 works where you know every like user account is in a way independent and they store and they can be like moved around between different databases like between different computers in the database and so. So this kind of allows to abstract out the complexity of the sharding from the user, right? As a user, if you go to near blockchain, you will not see shards.

We don't actually show them like you need to go to our PC and like query the block headers and stuff like this. Now the thing that we in 19 were planning to do was for security was based on challenges and that's proved to be very challenging. And this is across the whole space, right? We've seen like a number of other chains actually struggling with implementing challenges. And so kind of earlier this year we ended up kind of doing research and and refocusing on instead doing stateless

validation. So what this means is now when block is produced, block actually contains all of the state that that execute the transactions touched and that information is being sent around to everybody else. What this means is that other validators don't need to have state of the Shard, they can just validate the block on its

own. And it means we can have, you know, hundreds and thousands of validators validating every Shard. It can be completely random, they don't need to be assigned to specific Shard at any time.

And this also means we can you know have now a lot more kind of nodes and validators in a network kind of proving the whole system now kind of on a low level what it what NIR is, is really a decentralized shared sequencer that then sends out the data availability of this transactions across the whole chain. We use erasure coding and then we have this execution which now is stateless execution which then is being proven by number of other validators and and

settled right. So we kind of package the whole what now is modular framework actually in one you know pipeline way on top of the same set of validators, right, kind of just being rotated constantly across the network. And so that's you know at the core what near is. And so we actually going to be launching the new testing network for stateless validation to kind of as a part of our

phase two launch. And so this is kind of finalizes like core road map of of sharding that we've outlined since 2019 and this should be coming kind of January or February and we're going to have you know the full mainnet launch probably in April. And this is the idea that actually kind of conceptually if people read like with Dalek's end game this is in the way that structure you have block producers who are charted or kind of can you know we can keep

adding more block producers in parallel, so you can keep scaling the network. We also moving the kind of a somewhat in because of this block producers now don't need to rotate as much. We're actually moving the whole state into memory which gives us about 10X improvement on each shards kind of transaction processing.

And so like each Shard gets 10X and then you can have more shards And so then they kind of you know they do there's a coding date availability and then they do processing creates this blocks with state witnesses send them out and then you have large network of validators who who don't need to be this large who can just validate this blocks without having the full state of the chain. So that that's kind of the you know in a way finalizing our road map but also very much end

game. There's kind of bundles a lot of the current like roll up concepts and you know sells a base concept that is idiom is talking about into one product. And then we announced we we working on ZK Vazem with Polygon because this kind of just sending out state witness with the block is actually a lot of bandwidth. And what ZK Vazem allows us to do is actually to prove the whole block execution with state witness on the block producer

directly. And so now instead of sending like potentially you know MB of data we can just send you know whatever 10 KB proof out and everybody else can just validate that without re executing all the same transactions. So that is kind of actually you know final game.

I mean there's like a few more pieces that to complete the picture but that is the structure that we we think is pretty much final kind of architecture that you know you have tenship resistant shared charted sequencer, right. So and you can you know you have like all the data availability underneath to provide you so that and like we do data availability first before execution because that means all the other indexers and other piece of infrastructure can

start executing in parallel. And so you don't have latency on user interfaces before the kind of finalization of the execution on the validators themself. Then you have kind of execution on validators send out witness and now you know large network of validators can validate it and prove it without needing to have state for dated. And all kind of having like you know potential state is like 50 gigabytes for example. So they don't need to like have that 50 gigabytes on them.

They just receive whatever relevant for the transactions are being processed. And so that's kind of the, yeah, I mean it's it's a little bit complicated as a as a scheme but but like really it's powering this again like the kind of the end game structure that people been talking about at the same time it's it is like kind of that modularity just like reusing the same set of servers right to to ensure kind of throughput and latency low latency.

Yeah, like that's. That's like an episode on its own, to be honest. To dig through that, is it correct to think that like the stateless validation requires ZK wasam as a primitive? So no, because you you can do stateless validation without ZK. So what you do is you execute transactions, you record which pieces of state you touched and then you just send those pieces of state with with witnesses, right, with kind of proof that it's part of the state together

with transactions. And so we're actually launching that first while in parallel kind of working on ZK Wasim and so Zikiwasm, what it allows to do though is just compress all of these and execution of invalidation of this into just a

proof right. So in a way that Zikiwasm will prove the execution of this BLOB pretty much state plus transactions into just a fixed size proof but it but it's so it's more of an optimistic Zikiwasm from this perspective is optimization and it's obviously like way better for like longer term you know storage but it's not a prerequisite. I mean maybe I'll I'll try to present my simple imagination of like of the system.

So the way I imagine it is like if you imagine I'm a I'm a validator I'm an accountant right automated accountant. Essentially in near I have the capability I'm assigned somehow like some piece of work and and some of my work is also rotating right like it's not part there's a massive Ledger, massive Ledger, massive state and I am assigned hey go and make some changes to this part of of the state so I can I can basically

go to that part of the state. There are bunch of transaction associated with it. I execute the transactions and 1st I can I make the data available. Hey these are the transactions I am I'm going to execute. I make the execution, I update the state and today I somehow provide some witnesses so that for the other accountants I can sort of provide a proof. Hey, I did my job correctly, here's proof. And they don't need to download my part of the state to verify my work and the ZK proof will

make that even easier. So the state imagine as a massive massive tree or something. I can, I can modify some branches of the tree and I create a proof and then I that proof is witnesses today, Ezekiel wasn't tomorrow and I can send that thing to others. They don't actually need to have my part of the tree in order to verify my work. And and then there's a separate system that says OK, in modifying this part of the tree.

What are the transactions? I did some somebody duplicates duplicates that work and because because I can modify a part of the tree quite independently and there are many like me, so there are many accountants like me. All of these accountants are kind of modifying like different parts of the tree in parallel and like that is fundamentally why the system is able to scale. Yeah, very well put. So you have partnership with Eigen DA. Why do you need a partnership

with Eigen DA in that case? Yeah. So, so this kind of maybe, yeah, changing gears right. So so this is like near itself. This is near itself, right? Like it has no interaction with other things but. Yeah, yeah, so so and again Near itself right now is you know top use blockchain by number of addresses. For example, you know daily active, monthly active, weekly active. And so like Near itself has like a bunch of utility and and value already.

But again we kind of when we frame this like chain abstraction thesis, right. What it means is that for the developers and users on top, we're trying to provide a smooth experience across using other chains as well. And this is where we kind of looked around and like oh, NIR already has data availability built in like that's just part of our protocol. And so we have seen a bunch of layer twos that we can plug in into this to kind of hooking into the rest of our systems, right.

And so that's where we kind of you know started in in like kind of pretty much provided a way to hook in AP stack, CDK, Stark, Nets, kind of stacks. How do you publish your data on NIR? Now if you just publish data on NIR, it's useful it, you know, it's obviously very cheap. It's you know way cheap like cheaper than pretty much

everything else in the market. And because near as shorted you actually have more capacity than anything else can that can take your data already and we're going to add more shorts, but it's not as useful because you cannot route messages between between kind of smart contracts on roll ups between each other

and near in near contracts. And so that's where we had a partnership with Eigen layer not Eigen DA to help us actually do the work for this layer twos to get to executed state and outgoing messages such that the applications that want to route messages faster within one and two second, they can actually do

that through the near network. So Eigen layer validators will execute this roll up given the data published on near, they'll execute it and they will have a new state route for the roll up itself now. So think of it it's it'll be extra accountants, Etherium accountants who will be actually looking at the roll ups and updating statehood there, but then publishing back to the near like telling it to near

accountants as well. And so now near accountants and Etherium accounts together know the state of both near and all of the roll ups that are plugged into the system. And so now you can route messages between roll up contracts and near contracts and you know back and forth. And so this allows us to kind of again like align more the the,

the space of of the the space. And so again for chain abstraction for kind of aggregation it means we can do things way faster between all of the roll ups that that fit into the system. So that's kind of how like DA plus Eigen layer kind of provides this fast finality. And then you know there's other kind of tooling that we you know plug in on top is decentralized front ends to really kind of

abstract it from a user. But like we need that kind of alignment again neat near in the way each account like each each element of that tree is is separate roll up right And we kind of we have a system for managing them And so we kind of trying to fit the other roll ups into the same system and you know obviously we need to like plug in some pieces to to make it work under the same security parameters that roll up expect right which is a theorem security hands dagger layer and

then DA is kind of way to get this data you know into the system as well and and provide some guarantees there. Hard to unpack, but but like logically it's like, yeah, imagine, yeah, it's exactly that. It's imagine near as this massive tree and then there are like lots of accountants in near itself. There's one group of accountants and then accountants can kind of modify parts of the tree independent of each other.

We can send proofs about their modifications so that other other accountants can trust their work. And then kind of like this, eigen layer partnership is in some way saying that. There is CDM accountants, yeah. Yeah, it's like near, Near says. We have an awesome group of accountants. But if you want your own accountants and if you want your own roll up, you have created a

separate group of accountants. But then your accountants and the Near accountants we we sort of need to interface in some way so that so that the work, your accountant's date can be deduplicated on near and the other way around. And via this deduplication we can somehow achieve like trustless interactions between Ethereum roll ups and Near something like that, right? Yeah, pretty much like I would

say that. So the roll ups is pretty much I want my own accountant, right that runs everything. But then I I trust Etherium accountants to revalidate everything and and finalize it, right. So like Etherium accountants are are the final final. My accountant is the one who can do quickly, right. He sits right by my side. And So what we say here is near accountants can, you know can provide a bunch of value by, you know either connecting your accountant to the other guy's

accountant, right. So you can connect together or to our applications. But we still need to see them accountants because the finality of the roll ups is on Etherium, right. And so that's why we have Eigen layer pretty much to lend us their Etherium accountants to kind of use the let the you know as a roll up publishes the

Ledger right from their account. And 1st like we have the sedum you know accountants or Eigen layer to like validate everything quickly right before the fully sedum sinality will happen. And so that allows to kind of near accountants and to have like trust into into the execution of what happened on the roll up while also have the way quicker you know time to finality and to communication of messages for this roll ups and maintaining the same security as

as they have through ECDM. So that's kind of like, you know, it's like roll ups near an ECDM coming all together into like 1 happy family of accountants. I think that's, that's a great note to end on, right. Like big happy family of accountants. Yeah. Thank you so much for coming on. It's been like a massive episode, I think. Yeah, I need to like process this and I'm I'm sure our listeners will take some time to process everything too.

Well, we can do another one in a few months as we launched all this stuff, so. Yeah, totally. And yeah, we still also have Alex episode about the Smarter LLMS. Outstanding. So lots to do. But yeah, thanks so much for coming on. And and thanks for to our listeners, We'll have like 1 1/2 hours of of content here. Raise guys. Thank you for joining us on this week's episode. We release new episodes every

week. You can find and subscribe to the show on iTunes, Spotify, YouTube, SoundCloud, or wherever you listen to podcasts. And if you have a Google Home or Alexa device, you can tell it to listen to the latest episode of the Epicenter podcast. So go to epicenter.tv, subscribe for a full list of places where you can watch and listen. While you're there, be sure to sign up for the newsletter so you get new episodes in your inbox as they're released.

If you want to interact with us guests or other podcast listeners, you can follow us on Twitter. And please leave us a review on iTunes. It helps people find the show and we're always happy to read them. So thanks so much and we look forward to being back next week.

Transcript source: Provided by creator in RSS feed: download file
For the best experience, listen in Metacast app for iOS or Android