Tri Dao@tri_dao06:43TPUs have different bottlenecks from GPUs: few cores so you don’t need yo think about parallelism as much, large VMEM so easier to fuse kernels, but vector units have lower throughput so anything non matmul require much more care https://t.co/4ZdhowD46c#511482
Jason Wei@_jasonwei04:47@alexandr_wang Love this quote about Muse Wonder: "What I appreciate about Muse Wonder is that it isn't sycophantic. Some other models I've used will make an idea sound great in the moment, and only later do I realize it didn't hold up. Muse Wonder is willing to call me out and steer me toward a better approach, often before I've spotted the problem myself. That saves a lot of time, and it's a big part of why I trust it." — Jason Rothschild, MD, Musculoskeletal radiologist#511482
Jason Wei@_jasonwei03:31@kritika_singh24 So great to have you on the team Kritika, great work!#511482
Jason Wei@_jasonwei02:52In our health workstream at Meta we did a project where we wanted to see what research papers medical experts could write using a Muse Wonder coding scaffold that could access a database of 5M electronic health records we licensed from a vendor. The goal is that with 5M patients in this database, AI could do research to answer arbitrary questions like “did people who take statin have higher rates of diabetes later in life”? The thing we were betting on here is on what happens when you put human experts, a strong coding model, and access to data all together We worked with a small group of medical experts to help them use this AI tool and the punchline is that they all found that this setup enabled a much faster pace of health research. They generated ten manuscripts in around two weeks, with one researcher (Paddy Ssentongo) generating three. Throughout the process, they collaborated with each other to review manuscripts and write skill files to help our coding scaffold improve at doing research. The medical experts liked how quickly they could test different hypotheses and choose the best ones to continue working on Health research is pretty unique in that it is super important to be able to interpret noisy data and have taste to know which variables are important, which was still a significant value add from human researchers. Given that medicine is one of the oldest fields in human history and it is ultimately about humans, keeping a substantial human component in this type of research is probably important. In the near future I see AI primarily as a tool to facilitate human-led research. With the right scaffold I believe health researchers can be 10x or even 100x more productive using AI tools that have appropriate access to the wealth of data that the world has collected about human health One way to see this work is through the broader lens of AI for science. You can think of science as two parts: (1) collecting raw data about the world, and (2) synthesizing that raw data into understanding. In this project we already had existing data about the world and we aimed to accelerate part (2) using AI. Eventually, I imagine a world where we have such great AI tools that the time to synthesize raw data into understanding is driven towards zero. In that world, science will be substantially faster and most of the work will be focused on collecting the right raw data via physical experiments We did this in a pretty short amount of time (which was the point) and I think the future of scientific research will be more faster and more exciting than ever#511482
François Chollet@fchollet22:35You can't solve real-world physical tasks through autoregressive sampling alone. The physical world is entirely unforgiving to hallucination or approximate pattern matching. If we want actual autonomous robotics we have to engineer causality and discrete reasoning back into the loop. Biwei and the team at Aether AI is doing exactly the kind of foundational, non-consensus work the field actually needs right now. https://t.co/UJtL2t0tQ2#511482
François Chollet@fchollet22:28String theory is what happens when an intelligent system is starved of sensory input and starts hallucinating. Intelligence needs grounding. For this reason, intelligence is practically never the bottleneck to solving impactful problems. The bottleneck is iterated contact with reality.#511482
Yann LeCun@ylecun12:00@jakobmollerfi @JordanMich45361 @BachFrancis Yeah, like with multilayer nets, backprop, ConvNets, deep learning, and self-supervised learning. If you agreed with me you were clearly on the wrong side of history. 🤪#511482
Yann LeCun@ylecun11:57@michaelkheifetz @random_walker The real question is why they have such drastically different opinions from us. They changed their opinions. We didn't.#511482
Jeff Dean@JeffDean11:51@SumoTail @Khipu_AI @DeepIndaba @DiscoLoopAI Indeed. From a blog post I co-authored in 2018 announcing the establishment of our Google AI Research Center in Accra. https://t.co/ZT2oiCM8nD https://t.co/XYi18YiZcK#511482
François Chollet@fchollet11:43The ideal outcome for science isn't a couple of labs using proprietary models and millions of dollars to crack trophy problems, it's to have millions of AI-aided human scientists who follow their curiosity and become way more ambitious#511482
Yann LeCun@ylecun03:41@demishassabis My friend and Bell Labs colleague John S. Denker was the inspiration for the Mitch character in the movie. He got into Caltech at age 16, played a central role in a number of pranks (like winning a McDonald's sweepstake by printing thousands of entries using a high-speed#511482
Yann LeCun@ylecun02:54@random_walker These are all excellent points that I wish more people would know and understand. Particularly the people who should know better but are scaring everyone with predictions of cyber security apocalypse (in addition to job apocalypse, and plain apocalypse)#511482
Jeff Dean@JeffDean11:48@Khipu_AI @DeepIndaba @DiscoLoopAI As @NandoDF suggested, if you're at an AI company, please suggest to the appropriate person that these are worthwhile events for your company to sponsor.#511482
Jeff Dean@JeffDean10:59I've been a personal supporter (through our family foundation) of @Khipu_AI & @DeepIndaba, which are fantastic events for AI researchers in Africa and South America. I've encouraged Google to support these in the past. Discovery Loop (@DiscoLoopAI) is thrilled to support! https://t.co/MbaAheM76w#511482
#511482#511482#511482#511482#511482#511482#511482#511482#511482#511482#511482#511482#511482#511482#511482#511482#511482#511482#511482#511482