AI in Medicine Martin Shkreli https://desci.nyc/transcripts/ai-in-medicine Automatically generated transcript. May contain errors. [0:00] growing uh logarithmically or exponentially and this is the price of sending a message over the Internet so the point is that technology affords us all these wonderful things this is the resolution of a microscope some of these things are bound asymptotically you can't get any more resolution on microscope unfortunately uh well who knows uh maybe maybe we will be able to but as far as my little mind can comprehend it seems very hard to do so uh but compute seems to be on this Moore's Law that'll never end so why do we have things like uh that we're dissatisfied with right where is technology not working and I think that we know a lot about that there are a lot of sort of stubborn points in technology that don't seem to be affected this is a life expectancy so life expectancy life [0:42] expectancy seems to be on a negative sort of second derivative hopefully thankfully still a positive first derivative but there is this sort of like feeling like oh life expectancy isn't shooting up that much and I meet a lot of people in crypto if you're into longevity and it feels like we're not getting there at least with that kind of curve and what are the effects of we see so many things on the first slide I just showed that don't seem to translate into some of these other interprets for example National Healthcare expenditure has been growing very significantly since 1970 and I gave this talk that's a mighty League circuit that I did uh before my extended vacation and uh it was a uh it was uh it was interesting because Healthcare expenditures really started [1:22] boom starting in the 60s and it wasn't because of me I wasn't born yet but it wasn't because of an ancestor or mine it's because we actually crave health care it's something that we have virtually an infinite demand for uh Healthcare is something that absolutely no one in this room would say no I don't want to be healthy but there is a limit you have for other Goods right there is a point where you say you know what I don't need to spend more money on food I don't need to spend more money on on rent but healthcare's got this very special sort of place where we're willing to consume more health care at all times that's why Healthcare is on 25 of our GDP my guess is as we get wealthy and wealthier as a country and Society thanks to technology that Healthcare might even reach 50 percent of GDP um and it's because we all want a little [2:03] longer and we all want to be healthier but we don't seem to be getting that if we look at the amount of new drugs launched every year which is a decent surrogate I think for for health um if you think about the early part of industry uh I was going to say we but I'll just say bloodrock industry solved a lot of problems like infections the solving infections with antibiotics resulted in lifespan increasing dramatically at the start of the century uh anesthesia things like that there's a sort of dearth and then the Pharma had this Renaissance period in the 90s which kind of got me interested in Pharma around the time of the human genome uh decoding and form assault like virtually 80 of chronic diseases in my opinion and it was this Revolution and now with [2:44] oncology sort of being uh cracked you know there's there's sort of a question as to what's even going to happen after the end of that curve but it would have been nice if that curve went up and to the right just like some of the other curves did and there seems to be a problem with that even though the drug industry has gotten really big we're not getting more drugs every year which seems sort of strange even though we're spending more and more money so there's a lot of problems and I think that I don't know if I have a solution for you today but I'm going to take a crack at it um one of these pain points is to me actually it comes from the and as you may know one of the reasons that I became somebody that can get a room with people like this uh and not just some random you know startup guy or hedge fund guy uh that nobody would come [3:26] see is because I was very defiant and sort of mad when people got mad at me um the normal corporate reaction to a scandal is to do one of these and say I'm sorry and disappear and you know just sort of keep saying you're sorry but my reaction was was sort of different my reaction was no I want to explain why I'm right and you're wrong and I know more about this than you do and I'm going to take its task on every single point and that's sort of a losing battle in a lot of ways uh but it's a battle I was willing to lose anyway and I think most people would see that as a little crazy so I actually I certainly am not intending to blame anybody in this discussion but I want to point to the largest expense in in healthcare is [4:08] actually Physicians 10 of healthcare expenditures come from the farm industry 90 of the rest the rest of this in fact I think 100 of healthcare ultimately centers on a physician decision so the question is what can we do about that the number of physicians in our country has not changed very much and I think that the reason for it in fact this is a graph of the number of med school uh graduates every year it's very very limited uh we're not seeing more and more vet school graduates Physicians make about 350 000 uh per annum and uh it's a very healthy salary for for a job that increasingly we have to wonder if we're getting a value for that a lot of people feel that the AMA uh is a group that [4:50] tries to make sure that the physician stays Rich it's a group that tries to make sure that they can control the pool of doctors in the world especially in our country there are about 30 doctors per 100 000 people in America and again you can see that the number of new MDS every year is only about 30 per million so we're going to see sort of a prices I think in the number of doctors actually net doctors kind of dropping again which at a time where our demand for healthcare consumption is is just going up and up and up forever I think it's it's a really big problem so what's the solution I think that we could have a decentralized doctor I think that uh AI um basically can break the back of the lobby that Physicians have I remember [5:31] that Barack Obama asked doctors when he was coming around raising money uh at the luxury of eating him and he told me that there was a AMA dinner that he had and he said that he's going to lower health care costs and everybody clap I know he's a great speaker and then he said but you're all going to have to uh tighten your belt too room was dead silent you know the idea that doctors were going to tighten their belt for anyone is is sort of a scary idea and the question is like well how do we break this barrier the AMA is the group that make sure there's a certain amount of Med schools there are a certain amount of Med schools authorized to Grant the medical degree and that they have a certain authority to Grants so many [6:13] graduates nobody else can break into this without their permission and that's like the cartel that's the opposite of decentralization that's like the mafia like the dunes I hung out with prison to me something that's that's uh a great thing of course again I love doctors a lot of doctor friends uh at NYU there's the Martin shkreli professorship Nephrology you know so their doctors that have saved my life and my friends lives I'm not here to go against doctors but I am here to think about how do we get those step changes that I showed a slide or two ago where you know I used to have floppy disks and Now on my phone [6:53] I can store 100 gigabytes I used to you know not be able to sequence anything in a genome now you can sequence it in a minute all of these things are I used to be able to read all the literature on a subject and now I can't read one tenth of or 100th of the literature on a subject why are we getting that kind of stuff in healthcare and I think one of the big barriers is doctors so I am coming out uh with a product we'll talk about a second but there's other disadvantages doctors I do think cost and prices the biggest one but I do think there's a lot of variability reluctances live in New York uh in in other parts of the world forget America but other parts of the world you're lucky to get a good doctor uh at all I have a friend who lives in Idaho and he says it's weeks before he can get an [7:33] appointment I think that the amount of time you get with a physician is something that everybody has experienced with this sort of very quick you know sort of uh two-minute interaction of oh yeah you're fine I gotta go you know it sort of feels like a really not great experience uh to me I always remember to ask a question for the doctor 20 minutes after the appointment's over and I have to wait till the next time and by then I forget as well so there's also the limits of human knowledge and uh I'm happy to see so many familiar faces here but I have some friends in here that we've spent so much time studying one little space like we made this drug for this rare disease called uh pecan panothetic kinase uh Associated neurodegeneration it's very very rare [8:14] disease that very very few people know about even neurologists who are supposed to be specialists in this illness don't know what that disease is and I can't blame them um it's very hard to keep up with that medical literature slide that's going up into the right very quickly most doctors don't have the time to read the New England Journal every week let alone sit there on PubMed and and and go through every new new uh finding go to Every conference they're busy people they have to treat patients so it's really hard to define the limits of human knowledge so I've uh launched a product um called Dr GPT or Dr Gupta um oh my God well it's PPT with two more letters so I [8:54] figured it's okay um we also have Dr Lam which is a large language model so there's a few ways to slice it but uh this this website is off right now it's doctor Dr dot unconnect.io um and uh we've been training this thing over the last few weeks uh it's been a very hectic uh couple days in AI uh today some people joke too it's the Super Bowl for nerds when gpt4 came out and uh it certainly was cool uh but also the Facebook model was also leaked uh I think intentionally but regardless it's been leaked and is now available so we've been spending every waking hour me and nibbles trying to get this thing to work uh and uh we are basically trying [9:36] to see if can a doctor really be automated uh and I think the answer is conclusively yes I I don't have uh maybe I do have a graph for this actually that's it UI I guess for Dr Gupta but this is a this is a graph of uh basically a a test that the USMLE test but a group of other tests have been done as well and this is the performance of automated systems on that test and at this point uh humans score around in 80 uh extremely good doctor scores a 90. [10:06] it's about a 70 to pass the boards to get licensed as a doctor and Google's latest uh platform is scored a passing score uh to become a doctor in America which is sort of a big deal uh our product is free so uh it's available 24 7. uh of course we're going to try to make some money on it somehow but I think in general being able to give a physician's advice a quality Physician's advice to the whole world for free is something that could bend that cost curve and allow those resources to maybe be used more effectively I think that there's 22 million people in America that work in healthcare the actual provisioning of healthcare which is an enormous number [10:48] and the actual ratio of the doctors to non-doctors is kind of Staggering so I think that there are obviously limitations uh to an automated doctor I think there it'll work for some Specialties better than others I think that over time though my dream or vision is that we basically don't even see doctors anymore we go to phlebotomists we go to nurses we go to people like that and the computer will analyze uh our scores and tell us what's wrong with us and I think that if you look at the success of chat gpt4 released just today chat gbt three and a half which is what Dr Gupta powered on with some of our own special sauce um Chad gpg3 and I have could not pass the bar exam chat gpt4 scores a 95th [11:29] percentile on the board exam so very quickly and very scarily AI is becoming extremely good and uh while there's a number of regulatory barriers there's probably a number of personal barriers there's a number of areas of all kinds that people are going to wonder about AI positions I did a podcast with a comedy show last night and it's available online it's called voicecast and uh they had some like funny ideas as to why maybe you don't want to take your clothes off and from the AI doctor but regardless the um I think that you know the day of of basically relying on human Specialists is coming to an end we send a contract yesterday using chat gbt uh for a legal [12:10] compliment so ultimately I think whether you use this or not slowly surely the the age of AI is coming and we can use this to sort of decentralize and de-power against me this has nothing to do with blockchain I do want to make an API that's free it'd be amazing get a large language model on a blockchain somehow not sure I'm smart enough to figure that out but I do think that some people are working on something like that and so the more you can pull this out of sort of one company's hands as much as I'd like to have all that real estate to myself it's it's going to get more and more decentralized technology is a race to the bottom and um we've learned that from even open AI who has slashed their own price 90 just in a couple of weeks um so ultimately you know intact [12:50] sometimes you get it while it's good and then you have to move on um so I would have no problem if somebody made this and you know made it even cheaper even faster even Freer you know that's something that's just sort of natural in this in this line of work but I do think that's sort of one of the major solutions to to uh to uh Healthcare and um again no no disparagements to Physicians who work very hard and they try very hard but ultimately AI does things better than than us at certain Dimensions certainly it's a better chess player than any human chess player we shouldn't assume it's a better video game player in many different video games it's even now a better poker player which a lot of people felt couldn't be done there's no reason why AI couldn't become a better [13:32] position than a human and uh that's sort of a new product I have coming out soon so I want to talk about scientific compute very quickly a lot of people here probably heard about my project called drug light which I think is just sort of the start of what I hope is going to be a suite of tools that scientists can use uh scientific computational tools are very very hard to use you have to be a python expert you have to be a Linux expert you may have to be a kubernetes expert these aren't things anybody wants to be an expert in unless unless you want to be an expert in that but the average scientist doesn't uh the average chemist doesn't want to learn how to provision a cluster of AWS machines um that they want to be a chemist uh and and so the problem is you make these tools if you happen to be an expert at [14:12] that one thing uh and so if you we have this crazy list we show on another slide deck of scientific tools it's just hundreds and hundreds of random python uh libraries and they're really hard to use so we make front ends for them there are other companies that do that Schrodinger is a successful company in the pharmaceutical space but there are companies that are 50 times bigger that nobody's ever heard of like ansys Cadence synopsis hexagon are all bigger than GM but you've never agree if you don't see commercials for their products they cost millions of dollars and they're very very hard to use as a kid my dream was to be able to use products like that I forgot to mention it earlier but what really got me into computer programming was seti at home and of course Nobles uh but setting at home was [14:52] a distributed compute number there was a blockchain there was which actually came with some liabilities people would upload fake data on setting at home said he was the search for extraterrestrial intelligence and it was sort of uh it was done by now so it was a little bit inspired by the film called Contact one of my favorite movies uh where they take radio telescope data and everyone could sort of analyze this data little by little to see if E.T was falling home and uh so far they haven't found anything but ultimately maybe the weather balloons I don't know but ultimately the the that was one of the first distributed cooking projects ever and when I hear distributed I think decentralized and when I think decentralized they certainly think of D side so I think that we can make some [15:34] really great tools distribute them and get people to use them one of the problems that we've seen is that scientists don't just change their their workflows very quickly and that's something that we're sort of in it for the long haul to do that so druglike is our first product it's been out for a little while now struglight.com it's the only product like it's kind on the web so when I was a kid I always wanted to play around with this type of stuff I love chemistry I always wanted to sort of use a a tool that I can actually like I don't know make make a potential drug with or something like that it was always my dream to do that uh that's one of the reasons I got in the pharmaceutical business but there was no tool that was available for anyone I couldn't even hack or steal a tool these things were so hard to get uh that they [16:15] were just cloistered behind a couple of companies that could that could afford tens of millions of dollars of software costs so we put it on the web it's free there's no license agreement or anything like that and we also made freaking cute so you can use up to a certain level of free compute after that you have to pay uh and uh ultimately we hope that you know people will use or even abuse that Tool uh for their for their own benefit um if you just press refresh it'll give you another set of compute for free so that's one easy hack to to do that uh so we're making and we're in the midst of making molecular Dynamics tool which is a little bit different from ligand docking Lego Dynamics is when you simulate on you know depending on how you look at it [16:55] sort of a qm level of whatever bodies you're simulating um it's a sort of many body problem that's that scales exponentially as you add more bodies bodies meaning sort of atoms and this is a very important tool for both Pharmaceuticals but as well as Material Science and other fields uh we're working with a professor of fluid dynamics and circuit simulation is ultimately the biggest space in this is doing electronics and circuit simulation so we're making tools like that uh that hopefully will release over the course of the year uh we think that again the we're a for-profit company that hopes to make a lot of money but we also think that we're in a new era software where you have to let your user try your software hopefully extensively when you join AWS you have a free tier that's [17:36] pretty expensive you can host a whole sort of websites servers and all that for free and Heroku has a similar tool and we think that's really important if you are advisor and you really want to use our tool you're gonna have to pay us a lot of money but I think that if you are just a person a kitten in in a in your bedroom wondering about how to how to mess with chemistry we want that person to forge themselves on on our tool and learn a time and uh we're rolling out stuff like that over the course of the year I think that um eventually you know this stuff can move to the water chain I was talking to some of you earlier about how you know there's this crypto ethos about like purism that that I think is actually kind of detrimental uh I think we have [18:17] to take baby steps you know right now our product is on AWS and we're moving hopefully the next few months to adding this was web assembly and web GPU which means that we can use your actual computer to do some of the calculations and we wouldn't charge you for that we charge you because AWS charges us so if you want to provide your own compute and you have a bunch of computers you want to use you should be able to do that for free and that's something we're we're doing uh Latin work group is there's an argue that if you have 50 of your friends that also want to give you their frequency to your project you should be able to make your own little super computer and we're working on that eventually there should be a blockchain system these are very very hard to engineer we were just ate Denver talking [18:57] to people in the state of the art trying to figure out what is the best way to take a massive compute job chop it up and send it out to a whole bunch of untrusted nodes get them to calculate that reassemble them and send them back and get paid for to do that the same way people mine Bitcoin I think about these Bitcoin machines these vast arrays of you know uh Asic Miners And I sit there and say why can't we use that for drug development it's such a waste um and uh Bitcoin Maxi's sort of you know came out of that but do you think uh I do wonder if you know using a percentage of the world's electricity is uh to calculate random numbers is really a good use [19:37] um again I I think it's important to make it hard to hack Bitcoin but I also drool at the opportunity of of wanting to do this just 10 years ago we bought a very expensive software package from one of the companies it didn't work at all we couldn't even get it to run successfully without without it crashing and there was no distributed compute uh we had to talk to a company called prey that makes super literal super computers so the world's changed a lot in 10 years and I think that you know it'll keep changing so hopefully whether it's Akash or some other compute network uh there's a number of them like Golem uh and others that hopefully we can sort of find a way to solve that problem so I'll move a little more quickly there's a couple of other spaces that decide to Target I'm not targeting these [20:19] spaces but I'm certainly interested in watching somebody do it electronic health records is one of the craziest ones to me uh most people don't have a copy of their own EHR you should have 100 control of your EHR you should be able to give some a doctor read write access for half an hour right not have a company like Cerner or epic have controller for a hospital have control of your HR to me that's just totally crazy uh Regulators should focus on that instead of me uh the uh the idea that that you know we have this exact technology in fact zero knowledge is is hand in love for this zero knowledge proofs uh or the perfect tool for making reveal uh some of my [21:00] data you know I want this doctor this hospital to see some of my data but not all of it instead right now a lot of that data lives in a place that we just don't know where it is or how it's working and I know some people are working on exactly this problem in the DSi world so it's pretty exciting clinical trials expensive part of developing a drug if clinical trials were cheaper we'd have three times as many drugs we'd have possible products for Alzheimer's Parkinson's a lot of the tough diseases that haven't been solved yet how do we do it I don't know uh this is a really hard problem again Regulators in this case I think do a good job of protecting us uh you know but I know there are people my friends that are setting up uh ships that are just on international [21:41] waters so they want to do clinical trials in these ships crypto people are crazy the uh yeah and I know people that are developing their own medicine to take by themselves for just themselves and hoping that that medicine's safe enough um these are things even I wouldn't do but I think that uh I think that it's it's a it's an interesting idea to try to crack this problem of why does it cost 200 million dollars to study 200 or 300 or 400 people uh to to get an FDA approval um and the FDA is very rigorous it's hard to get people to do a trial without money doctors the first thing my experience I've run a lot of clinical trials the first thing they ask [22:22] when you want to do a clinical trial how much money is it going to pay and again I can't fault the doctor for asking that because ultimately if it's a money losing proposition in your position you may say you know what I have to sit this one out because my practice you know I'm used to this and I don't want to lose all this money by running this wonderful trial so it's not as altruistic of a role as sometimes we're led to believe um so anyway the group that can practice I know there's a lot of people working on it decentralized clinical trials and decentralized ideas even if you can't get something for FDA approval if we did a blockchain clinical trial of 200 000 people it's not like a real clinical trial but if it showed that let's say metformin which is drug and a lot of [23:04] crypto people take um would increase our lifespan by 10 that's not Class A evidence that the FDA would take but it's a lot better than what we have now which is just sort of guesswork so I do think the blockchain could do something here uh Publications I talked to a bunch of people um before this started that brought this to me and I think that this is one of the layoffs that we could do we have something called five which is the start of a huge decentralized movement there's a lot of sort of parasitic Publishers uh out there a lot of people don't like Springer nature also here companies that have made tens of billions of dollars basically on the back of scientists companies that now you work hard to make a paper in nature and now they make money off your paper nature they're [23:46] publishing something right it's like writing a book for free for you know Michael Crichton or your favorite author decided to write books for free and the published company made all the money it's very odd that we do this in science um so how do we do this uh without it um I think there's a lot of people working on this and it's it's probably going to be one of these size first major victories in fact if you exclude the kind of blockchain which I'd encourage you to do archive which is arxiv pronounced archive archive was basically the first major victory for that and I think that the problem with academic science is you need to publish a lot and you need to publish in the best journals and the problem with that is the pastorals are often run by a bunch of sort of older people you don't know that you have to [24:26] get into that club before they'll publish you in nature or in New England Journal or wherever the case may be it's really hard as a young scientists to even if you have a groundbreaking disorder people say that Einstein would not be able to get up today and I truly believe that yeah I could easily see that manuscript being rejected and we've seen a whole list of manuscripts that got rejected relatively recently over the last times one years there were groundbreaking ideas the Transformer paper which arguably is one of the most Landmark discoveries in computer science was never published in their own so I think that um self-publishing and things like that could lead to great success incentivization on replication if I'm a funder of research I might say you know what I'll get 50 000 for the the experiment but I'll give fifty thousand [25:06] dollars to the place that replicates it and I think that's uh something that could encourage positive behavior that we don't get expected so uh so why decentralize um there's just a couple of General themes those are all my ideas for sort of healthcare and science I think that you know some of this is relatively obvious you know we can do a lot with decentralization by hardening systems and it doesn't have to use blockchain but to avoid sort of single points of failure there are a number of ways to sort of focus it up I see that a bunch of people are doing Dallas for like specific things like hair now is I from what I understand a dow to cure baldness which is pretty pretty wild uh there's a number of other Dallas uh that go to [25:47] specific sort of uh goals and I think that's that's pretty cool uh but to me I think probably the best place to go is is is some of the stuff I like DB dial and others are doing where could you avoid replicating data that people have already done every day thousands of experiments are gone to different Labs without knowing that somebody's doing the exact same experiment down the hall and sometimes you wonder if you really need to do that those are part of decentralization is Distributing so I mentioned uh some of the distributed compute folding at home was the successor of the study at home folding home is a lot more well known and reached a lot more people but it was inspired by study at home and to the state holding it home and other at-home projects are being gone on a tributed basis so again these aren't necessarily [26:29] blockchain enable they do think they could suspense it and watch hands but um there is a lot of benefit to Distributing in decentralization and I actually think you could lower costs a lot of people wonder why you why do you want to do uh sort of drug or molecular interaction analysis on on blockchain I actually posited that you can actually have a lower price in AWS AWS has one problem it's a fantastic service the problem is Amazon Amazon needs to make a lot of money they make its fortune on AWS and that makes it very hard for uh them to give you a really good price whereas if somebody's willing to do it at Cost that's something that somebody might do on the blockchain and we've seen that with Bitcoin [27:09] uh democracization is something that that decentralization helps a lot with we mentioned archive briefly one of the things that people want to do is sort of democratize the FDA uh I think again sort of a crypto libertarian thing that's even too much for me uh I think that uh we probably do need an FDA but I know people that believe that the FDA is people and terrible and I totally don't think that I think they're great but I'm not just saying that uh every other part of the government's Choice words for but the FDA actually I think does does a fantastic job but to the extent that that those people it's not my choice uh to the extent that those people believe that we don't need an FDA and we should be able to take whatever medicines we want based on our own criteria not theirs then you know watching this is a [27:51] place for you and I think that people will be doing that democratization uh and again privacy is to me a part of democracy and you know being able to secure information through ZK is is pretty important so last but not least uh I want to thank everyone of course um I guess one of my my slides generally have like missing text but but I have a uh last message uh there's sort of this this weird group I've been a part of over the last year that's starting to gain a lot of prominence called Yak uh you may have seen it in my profile or something like that it's uh basically a reactionary movement to EA the post trial of a it's only a sandbankment free so so far we're doing all right but the uh the yak group uh is uh is basically [28:32] effective instead of effective altruism we're called effective accelerationism and I just want to introduce the concept in the group to to the greater crypto Community because I think they go hand in hand the thing about eak uh that most people know about us and we just did a call with Brian's of all people the other night um and uh we've had Mark Andreessen change his uh Twitter profile to uh accelerationist which which is a big win as well um our group is started by a bunch of people like Google and Facebook and other major tech companies that basically got sick of the censorship and the kind of the the fear that exists at these major companies about releasing technology and [29:12] I think that if there's anyone that could appreciate it I'm sure there are people in here that probably would agree with the EA side of it which is we have to slow down AI before it kills us all uh we think it's nonsense we think that uh technology is is what human Destiny is all about and that uh we want to accelerate uh AI we want to accelerate blockchain we want to accelerate every new Fringe technology that will become mainstream over the next hundred years for a better life for everyone and the yak movement uh certainly would welcome uh everyone a lot of us are crypto people some of us aren't uh but ultimately we have an anonymous leader who is a very very important person in technology who's doing a great job of sort of fighting the tide of of [29:54] Regulation and fear and AI there are people who are saying that uh a quick story is called Rocco's basilis the idea behind Rocco's basilisk because anybody show a hands for the story okay we gotta okay cool we got a couple people so ideas Rocco is going to come and get us because 100 years from now this superhuman AI which some people think will be three years from now this superhuman AI is going to take over the world and he's going to ask one question who amongst you did not want me to take over the world and he's going to take revenge of it and this idea is kind of chilling and in fact for a bunch of AI people it made them like turn their computers off and like take a vacation because it was so scary to them that that could actually happen that that person that AI that instantiation that [30:35] technology maybe it's gpt4 would ask itself which one of you tried to stop me which one of you said that I should not come here today and there are people of some senators in fact that actually want to regulate AI they actually want to say that if you have any kind of neural network you should deposit your code with the government first and that we should take a look at it and kick it around and make sure it's not gonna do anything bad and then you can use it and that's a serious discussion that people are having and so they sent crypto to some extent is about Liberation Freedom that's a very very scary sign to people like me who I play a very small role in this cult uh this uh a leader is is a very like I said sort of a a bigger [31:16] leader in technology I'm just sort of doppelganger but I think that you know we have to take care of whether it's blockchain or crypto and even Ai and make sure that the powers that be which is us make sure that the powers that we've elected understand that you know these are technologies that we want and need uh not technologies that that they're going to allow us to have and so it works the same way so anyway thank you so much for for your attention and time I'm happy to take questions I think we're gonna go into a place after I wanted this this fund has been really nice and letting us hang out here [31:58] for a while so I don't wanna understand is there any questions for Martin we'll do a couple questions just shoot it up yeah so with Dr GPC I think a lot of people stick with their pcps because they've developed a relationship over time with them are you thinking about like adding memory situation and like uh sort of doing that sort of thing yeah the hardest thing about large language models is they don't have State uh they're stateless uh basically uh stateless calculations and large tensors and they don't remember anything you can ask it the same question a thousand times and it will not you can tell us I love Coca-Cola I love Coca-Cola 8 000 times and then you can ask it what's my favorite drink I [32:40] don't know and and to me that's it shows how naked gbt is and it also shows I think the EA AGI safety Community how stupid they are so because ultimately these things aren't that smart uh and they're not going to get that smart anytime soon so I spent a lot of time I actually published an AGI written book which is really hard to do all right I'm sorry a GPT written book uh on Amazon uh so you can take a look at that if you want uh it was really hard to get this book to work uh so we're a long ways away from from being able to do some of these things I do think that Dr Gupta will become your friend uh but ultimately it's up to us to make that work and I I again if I can take five [33:20] percent of healthcare costs and lower them that's 200 billion dollars a year so ultimately Google took us the advertising Market which is fairly small and made themselves one of the biggest companies in the world uh by doing something like that in healthcare you could do really well and help off people especially people in other parts of the world that getting a doctor is really hard gbt is fluent in every language which is kind of amazing so I hope that it can start to get a personality we've programmed this meta state that sort of keeps track of some of the things you're talking about out and then like kind of is a controller of the llm so the programming is much more intricate than you might think you might look at Dr gupt and say oh that's cute you took the GPT API and you connected it to it no it's a lot of programming and it's really hard to do because it's stateless [34:01] and uh again the the people at open AI did a really good job with chat GPT because they they did a similar thing where they had to sort of help the conversation remember itself uh so it's a very tricky programming thing it'll get easier over time and I think even with GT4 you're going to end up liking your doctor anybody else yeah I mean a little bit picky backing on that for the last hundred years you've had people worried about this new Tech's going to take away our jobs since the Luna attempt maybe it's better to think of it as a compliment rather than a substitute to your doctor so I mean I see it more as instead of having 10 doctors I have one doctor and this but I'm still going to see my doctor the doctor who's in my personal relationship [34:42] but the expertise is going to be this GPT which is probably better than that Doctor's memory I don't know so maybe like in terms of how you Market this product turning that statement to a question do you think that really it's not about antagonizing the position but absolutely some Physicians on board of having Physicians leave the charge we just downloaded to more or less all kind of ring we got a rave review um one of the top uh neurosurgeons there said that he loved to use it uh not because it's smarter than him but because yeah I've been doing the journal every week since I was a teenager and it has a forgot or worse it has a uh every week There's a clinical case from Mass General where it's like a mission it's like Dr House in real life and they list [35:22] all the The Narrative and at the end they say what happened and I fed some of these cases into Dr Gupta they're extremely hard to figure out uh and one patient had ursulosis which is insanely rare infectious disease doctor GPT figured it out before the real life doctor did that so occasionally you're going to get that win a lot of times doctor to go up to didn't figure it out but the key is that one time where doctor says man I got this guy dying in this ICU and I don't know what's going on I wrote this thing in that GPT thing let me know if it works and if the the gbt solves the case one out of ten times listen it doesn't cost anything and you know those kinds of clinical assistant tools what we're calling it uh it's also helps on the regulation side [36:02] um that uh these uh these things could help a lot and again I think one of the spaces that people are really excited is taking on which a lot of people I think virtually everyone agrees uh is a space that nobody would cry it went away is a legal space so yeah it's a bit different from what we're talking about today in science but [36:27] yeah we thought about that a lot too um that's why we're starting a company in an island called Vanuatu uh no it's it's a tough question I think that you know there's a lot of questions about this llama was leaked by accident the other day uh by Facebook and it's uh on 4chan now and it's a fortunate to put it on touring and now the whole AI Community has been been wrestling It To The Ground uh over the last day a few days and people have asked well were we stealing this you know what's going on there and uh we won't luckily decided that we're not because it's just a list of Weights those weights can't be copyrighted or patented it's sort of a um one of these things where I think [37:09] technology leads the regulation out the other way around uber is completely legal right Uber's against law right why would you have a taxi Medallion service if if uh if if it were all legal or why would you allow Uber to exist if you had people pay millions dollars for taxing medallions taxi Medallion New York is now worthless uh Regulators could have shut it down and in certain states and cities they tried but the same Regulators on their way to the meeting to shut down or go into the river and deciding you know come on wait a second what am I doing you know this technology is so good I can't kill it off so what we're trying to do incidentally is we're going to try to do a double-blind study where we put 100 doctors in a room this John searle's Chinese room experiment feed them in the clinical case and they [37:52] have to feed out the clinical case and text and we do the same thing with Chachi petite then we get a impartial panel of doctors to score on double line basis which came from which if we can beat the doctors I actually think it's about practice not to use it because ultimately if we can prove it's sort of like self-driving cars Elon musk's full-service drive is not going to be Flawless it's going to kill somebody by accident but we have 50 000 Automotive deaths in here so if that drops to ten thousand I'd mourn all those 10 000 people but at the same time I'd say we save 40 000 and ultimately which is the better of the two clearly I'll take full service drive and we shouldn't be you know shooting Tesla every time somebody gets into a full service Drive accident so I hope that malpractice will drop right doctors aren't perfect the 85 [38:32] year old Doctor Who you see might right the wrong uh uh prescription Pat on uh or just by having sloppy handwriting which all doctors are joked about having is going to write the wrong thing so to me I think that the imperfections of AI are going to also result are going to match uh and hopefully be less on the imperfections of humans I hope that helps out so I don't know is assertive but what do you think is the role in the place of a project like folding at home in a world in which I'll perform exists does it complement Alpha fields in certain scenarios or for certain problems or instance the protein solving problem yeah it's a good question I mean the all these things still require some compute and somebody we had a talk with the zero knowledge expert today who said [39:13] oh you know I talked to a bunch of scientists and there actually isn't it's not a scarce resource and I I sort of you know had to be really polite because I think there are things like there are certain uh algorithms you can run that take almost no computing power uh then there's stuff that we can't even imagine like a planet scale idea like for example could you try to simulate I don't know like uh hundreds of thousands of albums like wholesale it's impossible uh you could you could do some meta simulation but to actually simulate each molecular interaction this is impossible what scientists like to do that you bet you know that would be amazing uh so how could you actually do something like that so for alcohol you know folding is the shape of the protein is just one small step I always get people who they're not intentionally trolling me [39:53] but it feels like it when they ask me if the drug light is about folding proteins folding protein is the making of an algae on the fly but it's the first step of setting up the easel after that you have to decide which molecule fits into that fold so you fold the protein in it it's a three-dimensional structure and you can quibble about what that structure looks like it's very important you know in the traditional you'd use x-ray crystallography with folding uh with Alpha fold you can actually do a computationally great whichever fold you use now you need to do the heavy duty lifting which is what the front light does I'm curious what the competitive space for a gun life is yeah so so yeah I mean we're up against a lot of older companies that have software that [40:35] is impossible to use so this is a very small space drug like target market like the hope for Revenue someday for us is about 10 or 20 million um I've had drugs that sell more in like a week so it's it's very much a you know a labor of love you know I think it's an important tool but it's definitely not the only tour yeah you should be scared of you then I think Schrodinger is is who we're sort of coming for Schrodinger makes the best tool it's awesome it's just a hundred thousand dollars you know per person per year and after that you need to pay for compute they don't give you a stitch of compute so you might pay millions on that and you got to pay for integration so they don't like have a connection or [41:16] a pipeline type committing to us so it's a really tough system I hope to eat their lunch you know I'm sure that they'll be fine but you know we'll see so what's your future vision of Dr gbt is it become a generalized to that people just ask the question or it become a customized to like I can upload my history of medical history taking knowledge to the doctor GDP and it kind of gave me more personalized yeah no it's 100 personalized we do want you to upload your history medical history he'll actually ask you over and over again so if you try to change the subject he goes listen I don't want to talk about that and if you person and he'll say I'm sorry that's inappropriate [41:56] but let's he just drives you right back to the conversation of what medicines are you on how tall are you etc etc and he stores that uh and and then from then on can give you the personalized information for like an advanced service people are really interested in their health so let's say you have migraines I have a close friend that has terrible migraines and she wants to know if anything new happens migrants we could have something like doctor whooped up every time you have an appointment with them which is whenever you feel like he can say hey by the way it's a new research migraines this could be something that you're interested in there's new medicine there's a new something and that can kind of be more interactive tool where it feels like your your doctor is like limited resource where you could ask any question or how to explain anything to you [42:36] anyone else uh who's the buyer for Dr Gupta question uh so what's amazing about GPT is how cheap it is so we've been running this thing uh non-stop for several days and I think our API bill is like seven bucks or 17 bucks or something crazy like that uh and we're using uh 20 different computers at Amazon to run it and it's that bill was like a thousand dollars so to me I'd feel very weird like forcing people to say oh no you got to pay a bunch of money to use it I think that ultimately the buyer's the average person and a physician so you could have like expert mode where it's doctor to doctor hey Doc I have this case what do you think I was a doctor I think that you should do this and that [43:16] could be a doctor pay like 100 bucks a month or something like that for it and for the average person I really do want to make sure it's free uh I think that my partners and investors probably want me to make it so that you can talk to it three times a month and then maybe the fourth time we want your credit card number and the fifth time you're gonna get a bill and you know that kind of thing the traditional SAS kind of business but ultimately what scares me about that is that anybody can make I think something like a doctor who I think harvest the best and we're training our own models and we're doing all kinds of stuff that'll widen our mouth but and as far as I know I asked around the AI Community who's doing this and they named a guy that works for me so the uh yeah so so far so good but we'll see if [43:57] we can keep that load but the nice thing about decentralization is that no capitalists can can like go into technology thinking that they're safe uh the same way Schrodinger thinks that they're safe and they're going to dominate of the ligand docking molecular modeling space uh you know every company even Google is now looking very vulnerable to open AI so do you think that average clinician uh trusts like the average person enough to like just take you know their like text driven input and make a diagnosis for me because I feel like when I go into my doctor like to listen to what I say and be like cool uh let's run these tests and like let me give you your real diagnosis like from the test themselves uh so I'm wondering like how you see [44:38] that interface going right like if you're selling directs to either the average person like how do you deal with this like data augmentation that like uh actual clinician would have I think there's a subset of clinicians such as humans in general that just have this kind of like negative attitude uh and it's just part of outside manner people call it or bad customer service but my dad just went in for uh prostate cancer scare and uh he said that he just couldn't get a question into the doctor you know doctors was hurting him off and you know disrespecting him basically my dad's uh you know uh you know relatively smart and relatively fluent in English but he's he doesn't come cross-grade and he the doctor just shut them out of his way and I can imagine what it could be [45:20] like sometimes maybe as a menorah or maybe as uh whatever somebody who doesn't speak English going to an American doctor and getting treated like dirt to me that kind of person deserves the same health care that that I do when I go into the doctor's office I tell them what to do but the uh the uh the reality is that how informed you are is a function of how much respect the doctor May give you and a lot of doctors won't give you respect regardless WebMD is the real answer that I'm trying to say here which is that people go on WebMD before they go to the doctor and convince themselves they have interstitial pulmonary fibrosis and the doctor's like you got a cold come on you know and I think that you know we all know somebody like that but ultimately I I don't think more information is going to be punitive um I do think that my goal would be to [46:00] turn the doctor into a relic that the only doctor left is a surgeon and even that could be done by a computer and arguably increasingly is so I think that slowly and surely when I get really mad at doctors uh I say kind of a pretty pejorative thing I call them uh very overpaid plumbers um but it's uh respect for the comments the uh it's very hard to fix a toilet but no in all seriousness WebMD I mean if you've ever studied treatment algorithms like hypertension you say Okay a patient is doing the attacks then do y patients why then do Z and it's fairly cut and dry and I think doctors have this very resulted place in society pardon is artificially created and I [46:42] wonder if like through Chachi BT we can go to doctors and say wait a second doc like my dad uh for instance doesn't have prostate cancer thank God but the doctor wanted to put him through every test imaginable and I told him Dad I think that doctor's just trying to rinse a couple dollars out of you I don't think you need six MRIs I don't think you need a prosthetic with me I don't think you need this and that but and he's a really important because he scared the out of me and I said listen I know prostate cancer and you don't need this stuff but a Chad GPT not everyone's son is the farmer bro but if you have a champion gbt you could ask it and say do I really need this or is this doctor just kind of with me and I think that second opinion role could move it too I don't know if that answered the question but I did yeah how are you thinking about using or not [47:22] using people's data who are kind of querying everything yeah this is probably some lightning questions so just like yeah it's a really hard question I sorry about that I did pick a bar out I automatically forgot its name as soon as we figured out do we remember the name of the place that we can go oh we have another talk doctor so oh yeah okay there's a couple more questions over here just want to make sure everyone gets their question yeah so it's hard like I I don't want to go to jail again so I think that storing people's data in a compliant way that's works with the blog it's a really good idea uh I still think there are people that are going to say you know I don't care about well I I want this advice and there are powers that need [48:03] that are going to say this is a medical device this has to be regulated Etc et cetera and that battle Hib isn't a law for a reason and it should be a law it's a good law uh so we have to make sure that one of our programs can't just jump in and see his ex boyfriend or girlfriend's data and say oh you know look at that I didn't I knew it uh you know so that would be a terrible terrible thing and that happens at companies like like uber and other and Banks and things like that but you know it's it's going to be important that we don't store that information unless they want us to store it and we can't see it unless they want us yeah uh I got another question to that point about data collection right like is there a minimum threshold that you need to know about a person and test that you need to run like not to frame [48:44] this like in a theranos way of like oh we got we need like one input of blood but like I think to the point of Health disparities that's going to be driven by you know even gathering information about your hype or like your blood pressure a lot of people don't have those tools like how much data needs to be gathered by a doctor and perhaps how could you work with the doctor to gather that data and then yeah no that's that's one of the revenue opportunities to somebody's question about a customer is is eventually there's gonna be questions Dr Cooper can't answer in fact immediately and I think one of the ways we could do it is kind of like because saying hey you need to go see somebody in your neighborhood do you want to see Dr Smith by the way Dr Smith's paying me [49:25] 20 to tell you but do you want to go see Dr Smith tomorrow at two o'clock and I think that's something that could be a good you know kind of a Matchmaker and help people get their health outcomes but it's it's a good question I mean I don't think that you know there are elements like psychiatry you probably doesn't need to see you in fact most psychiatrists are moving to tell them so that's an example of a place that'll work really well ontology I'm not so sure right like I said everything is a little bit different neurology seems like it'd be almost impossible Dermatology also seems tricky so like there's a bunch of things where Dr Kai needs to see you so I don't think doctors will go away to the dinosaur but I think even like I said five or ten percent Savings in healthcare would be incredible when well thank you everyone [50:07] I really appreciate it you have a gift for you so so you can see it but we made you a Eastside NYC teacher wow thank you yeah and um you are the first so you can also see it has the Statue of Liberty with a little feature holding it um thank you um there's a couple things if anyone we have some books in the back we had a book signing last night for two of the books I wrote um if you want one you can have it and we can sign them after and [50:48] also if you want a mug or if you pre-ordered a mug um just come up to me after and we'll get to your mug with the D size stuff on it and then okay so um like what Martin talked about um was like it was like clinical trial data and how to collect it and like that would be one of the interesting things around um decentralized science so I'm going to talk about the stuff that DB dial is doing to um help people collect clinical trial data so the first thing is is like what what is what is DB now and it's kind of like an air table for web 3. so here's [51:29] like a quick demo of how it works I might need to tether the data right yeah so it's like a web it's like an air table for web it's free from Wi-Fi here connect here we go all right this work now so this is just like a little screen recording of how it works so if you're like a clinical trial you first like create a table of what you want so in this example um we're just going to say that [52:11] um we're like a butterfly we want to say say we want to collect every butterfly in the entire world so this is like a very distributed food and the way you would want to do it is you would want to create like a form and then send that out to people all over the world so that they could enter in that the butterflies that they're seeing I think this is like a cool example of like where decentralized science works so um we go here so we create a form around butterflies of the world and we say the first thing we do is we say like we want to collect all the different species so we create a row in the database for that and then um up at the top you can see when we create a row um we call them there's also these other [52:52] columns here around nft um block Explorer and ipfs so this isn't like a normal air table it's something where when somebody submits their data to the database they're not just lobbying that over a fence and that's the last I've ever seen other data they're creating an encapsulated like data packet that lives on ipfs and that they can show that they own using an nft and that nft is then part of this larger data set so you're so and if when you go to your wallet or when you go to openc you can see every piece of data that you've ever submitted and this is like a very cool thing like when you go to when you've submit data to Facebook you can kind of lose track of it but here you [53:33] just go to your wallet and you go to open scene you can see every piece of data that you've ever created online so we have these different things here um and what you do is um you have IP pass whatever so now we um we create a form for this so we want to be able to have people populate this so we say the form is called Butterfly form and then we get this form and we send it out get a link to it and we send it out to all the people that are interested in it so I'm from Santa Cruz so I would put the Monarch and that submits it to the data set but you can see here that Monarch is actually in my wallet so this is open seat and this this is a living in on on [54:15] the blockchain the piece of data that I have contributed to this data set so you can also see it on walk Explorer and you can see it on ipfs so this is like a really simple primitive but I think it has some cool um like how it's being used is pretty cool so like Martin talked about here and I was using this to test out clinical trials on different hair loss drugs and this is like a really um game-changing way of doing a clinical trial because it's not about going through the FDA or the drug that they're actually testing out went partially through FDA but then kind of got sidetracked when another company French company bought it out but there's this drug called ru58841 that's look that [54:56] looks fairly promising that doesn't have the quite efficacy that would be needed for FDA approval but they're re-picking this back up and they're using dbdao to power their whole data collection process so you can see um their form they created this whole patient portal here where you enter you upload your whole genome file on the VCF you you upload your treatment report which drugs you're taking how it works you can upload photos all this is stored on itfs it's encrypted using keys on your own and so this is like a very new way of doing it and the other thing that's really interesting about this is like uh like Martin was talking about it's much [55:37] cheaper so they're sort of leveraging the fact that um there are these communities on Reddit of hundreds of thousands of people that all will almost for free or because they want to cure to some sort of problem that they are facing they will contribute their data um and in many ways they contribute it um not under traditional HIPAA or like uh HIPAA like things they just want their data to be public so that they can find a cure for this stuff and they want to be recognized for that which is which is a different way of thinking about uh how traditionally chemical clinical trials are done another one that we're working with is this um company called altaflora so also floor is like a traditional web 2 [56:17] database or web 2 clinical trial company and one of the things that they're they're doing long coveted studies for people that want um help with long covid they're also doing things that would otherwise never get through FDA approval so the one that they're interested in is getting CBD to kids with epilepsy so this is not something this is seems to be fairly promising um based on sort of uh what I'm reading online and what they're telling me but basically when you give a kid's CBD who's having an epileptic seizure uh it basically like subdues them and helps secure it but this would never be able to go through FDA trial so they're trying to do this on their own and I think that's like another way in which like their traditional web 2 company and sort of we came in and just [56:58] like swapped out their back end and now they're running uh they can run on uh to be down [Music] um I think there's some other cool things about like why like web3 needs a database um you're incentivizing people to create data so that um you know right now you have no incentive you you're when you've put data somewhere you don't get anything back and I have this concept of like negatively price data so normally I put something on AWS it costs me money but what should happen is AWS should pay me for the month for the data that I put on them when I monetize what as they monetize it so that's sort of like what you could do with like a web 3 database where you can have other people monetize your data um data permanence this is a bigger problem [57:38] um in like academic research basically like a server will crash a lab someone won't get tenure in a lab grad students are always doing other things so you have academic research and a lot of the data just sort of disappears this is a bad idea um but when you put on the blockchain and you have a universal naming structure it sort of always lives forever um another one that is problems with problematic with existing uh databases is you can't link between rows very easily so having a flat main space where you can link between different data sets what I think will enable new types of inference you can encrypt things it's tied to your identity there's just a lot of benefits so the way it works is like you put in [58:19] your data into the data set it has to be accepted into the data set so someone eventually you put I say I have an x-ray I submit it to the database and then it has to go through a curation process so the whole database is actually run by a Dao and the Dow is a multi-sig that either accepts a piece of data into the data set or rejects it based on the correct their criteria but once it's accepted into the data set you're then entitled to the rewards that that database generates so let's say you have a bunch of X-rays and some lab wants to get access to those so they purchase them all so they'll purchase the whole data set for a hundred dollars then 30 will go to the person that curated the [59:01] data set so they'll get thirty dollars and then the other seventy percent like goes to the people that contributed the data so each person will sort of get if there are 70 people that contributed each person will get a dollar every time that data set is bought for a hundred dollars and this allows for like your residual data and the other thing that's really cool about this is like um everything's like an nft each piece of data is as a token so you can start applying all the normal defy protocols to data now so I can lend out my data I can stake my data I can get a loan on my data based on the cash flows that it's generating [59:41] um everything is is encrypted so we can encrypt columns or rows using this thing called lit protocol which basically allows you to encrypt certain areas of it so you can that's nice and um yeah the business model I talked about basically like when revenue is generated it's split between the various people that created the data and I think another interesting thing here is to look where the underlying data the underlying value comes from so traditionally like one piece of data isn't very useful but when you put together 100 pieces of data you created something that's more than some of the parts and normally that more than the sum of the parts would just go to Facebook or Instagram or someone else but here we're taking that excess [1:00:22] information it access value that's been created and redistributing it to the people that actually contributed to the data set so that's like where the value is coming for the person that is buying this data set um yeah uh if you want to work together uh we're always open to working together with new projects we're working together with some sociology projects we're working together with some insulin projects we're working together uh perhaps with uh bio and uh molecules and stuff so we're always looking you can just reach out to eat out on XYZ contact or my telegram is uh my Telegram [1:01:03] is right oh my telegram is here go go go Michael um thank you guys I appreciate it anyone have any questions here's Eva yes yeah I was curious on the I'm really interested in that kind of the the monetary split and how you do that and is that something so I get that like you guys would need like 10 or something but is that something where like the the split you you made sounds kind of good in general but can individual projects yeah those are example splits yeah yeah that so basically those splits [1:01:45] um will be written into the smart contract so yeah you can't be like rugged with the terms of service update right right cool yeah but those are examples yeah yeah that are not set in stone yeah I could see different projects having different exactly kind of like the Scout in some cases there's a lot more work yeah others maybe yeah yeah yeah yeah yeah yeah how will you verify that the data is not like that yes so that's the job that is not what like oh yeah so that's the job of the curator of the data set so the curator of the data set checks to make sure data is either good or bad before guys in the back um the person who's the curator checks to make sure that the data is either [1:02:26] good or bad and they need to have domain expertise to make sure that like they understand it so if it's like x-rays those persons that person needs to know if it's an x-ray data set or if it's like sociology research that person needs to know I mean you need to know in many ways you can check the data you could verify who's who where it's coming from so it could be like a white listed group of people that submit it um it's very dependent on types of data sets yeah Francis has been like a lecture or two about data and health and genomics wasn't anywhere in there um well hairdos I heard I was doing the VCF file here okay um yeah so they're they're they're uploading your genome [1:03:06] file and they're extracting out um various parts of this skills a little bit they have this they've really thought this through um and uh we powered the back end but they we actually had them come talk here one another time and um they they know a lot more about the genome stuff so I just gotta say when you when you go to the the literature like to find all of these like smps related to different diseases and stuff like trying to actually decipher what that actually means even when you have the VCF it's very difficult oh yeah I mean this is a next level one built from Top I guess yeah someone else that's another little abstraction anyone else [1:03:46] oh yeah is there any way to deal with like curator bias uh with the data sets or is it they have the ultimate decision on whether data is getting out of their remote yeah that's a great question um basically every decision is written on chain so you can look to see if a curator is systemically rejecting certain pieces of information and you can see like what's assuming it's not like there's different things if it's encrypted or not then you have an issue but like um generally every decision to accept or reject is an on-chain decision so you can in theory write a chat GPT bot that will read over every single decision that the curator has made to accept or [1:04:28] reject pieces of data and re give us credit score sort of to the curator bias and what's cool is you can have a chat GPT bot that you make and I can have a chatbot that I make and we can get different scores which is which is I think is a healthy thing um and yeah so I read it's like it's kind of like case lawn like the legal system like you can look to see every decision a judge has made and determine if that person if that judge is biased oh then that's just the people's movement so if you shouldn't participate in a data set if the curator is biased and you should start your own data set that is free room yeah [1:05:11] it's not slashing it's what you're saying yeah you just you just go to a different data set that's not biased yeah cool all right well um thank you guys again and feel free to just mingle and talk and then we'll go to a bar after and um yeah again thanks for coming out and we're hosting another one um on April 12th I think I'll double check and I'll send out information about that in a follow-up email but yeah thanks again for everyone for coming thank you here oh my God yeah [1:06:15] thank you [1:06:31] the video beginning because um