Although the
Stepping through the 70 years of history since the field of study began, we arrive at the form of AI that has garnered considerable attention today: Generative AI. This term refers to models capable of creating content such as text, images, and audio. Generative AI became more prominent with the development of advanced models like Generative Adversarial Networks (GANs), which revolutionized the ability to generate realistic content.
The emergence of transformer models, introduced through Google's "Attention Is All You Need" in 2017, represented a pivotal moment in the advancement of natural language processing. Today, transformers have become the focal point of discussions in AI. Although OpenAI was among the first to see value in transformers, well-capitalized teams have started training massive transformers using vast datasets and computing to create what we now call LLMs (large language models). The effectiveness of this architecture became evident with OpenAI's launch of ChatGPT in late 2022, which quickly emerged as the fastest-growing consumer internet app of all time.
"I sometimes see people refer to neural networks as just ‘another tool in your machine learning toolbox’…unfortunately, this interpretation completely misses the forest for the trees. Neural networks are not just another classifier, they represent the beginning of a fundamental shift in how we develop software. They are Software 2.0."
Golden Age of AI R&D
2023 saw huge leaps in the advancement of AI – dive into a few key highlights below
2023 MODELS
Significant progress in AI has been made in both closed development and among the open-source community.
Proponents of each side share lively debate and ethical concerns regarding the development process. In May 2023, a leaked internal Google document titled, “We Have No Moat, And Neither Does OpenAI” went viral for suggesting that Google and OpenAI were at a disadvantage in the AI industry compared to open-source AI models, which were described as "faster, more customizable, more private, and pound-for-pound more capable." Open-source models enable rapid problem-solving and distribution of solutions, as seen in the modern internet's reliance on open-source technologies. Closed-source models, on the other hand, offer proprietary advantages that include stability, focused product development, and accessible customer support. These models often feature advanced functionalities and built-in security measures, catering to organizations that require enterprise-level features and support.
The battle between big tech companies and the open-source community is not just about technology; it's also about the philosophy of knowledge sharing versus exclusivity. While closed-source models can provide competitive advantages and revenue through licensing, open-source models thrive on community collaboration and the free exchange of ideas. This dynamic creates a balance between accessibility and exclusivity, challenging both development channels to continually innovate and adapt.
"The battle between
Alex Rampell, Andreessen Horowitz
In the current dynamic landscape, incumbents recognize the
The ability to take advantage of this new technology via API empowers incumbents with the agility to swiftly adopt cutting-edge technologies without the constraints of traditional development cycles. This phenomenon not only enhances the incumbents' ability to adapt but also allows incumbents to take advantage of their extensive market reach and private data to augment lower quality public data.
Generative AI startups raised nearly $50 billion in 2023 according to Crunchbase. And its clear value will continue to be captured by startups in the age of AI. It can be hard for incumbents to overhaul from a traditional enterprise SaaS tech stack to an AI-first tech stack. Consequently, incumbents are increasingly reliant on startups for the infrastructure to build AI apps. Some examples include:
- Model development companies such as OpenAI, Hugging Face, Together.ai, Anthropic, and Mistral
- Frameworks such as LangChain and LlamaIndex
- Vector databases such as Weaviate, Pinecone, and Chroma
- Evaluation companies such as BrainTrust Data
AI Enables
The advancements in AI infrastructure are setting the stage for innovative applications that redefine industries and user experiences. We are starting to materialize visions for how AI will reshape various industries and aspects of our daily lives from how we create, build, and take in information. A broad variety of exciting young companies are tackling issues through the use of AI.
AI is democratizing creativity, as new applications enable individuals and businesses to harness powerful tools for content creation, design, and multimedia production. Pika Labs and RunwayML enable text-to-video generation and Midjourney is an AI art generator that creates images from text prompts in seconds. Descript boosts podcast and video production with AI-driven editing tools and Synthesia enables businesses to create professional-looking videos using AI avatars, democratizing video production for enterprise applications. The ability to create extends beyond traditional boundaries and into the future of software development.
AI-powered coding assistants and next-generation development environments are reshaping the approach programmers take in conceptualizing and implementing solutions, while also broadening the scope of who can be a “developer.” Cody, an AI-powered code editing assistant developed by Sourcegraph, is designed to help developers build better and more efficient software by providing context-aware code explanations and assist in writing, reviewing, and refactoring code. Cursor.so is an AI-based Integrated Development Environment (IDE) that offers natural AI-based workflows within the IDE, such as ChatGPT-style code chat, inline difference of AI edits, and auto-debugging by hovering over errors. And Replit is democratizing programming by enabling individuals with varying levels of technical expertise to leverage AI for coding, allowing them to build end-to-end products and prototypes.
Online search, which has been largely unchanged for decades, is also being transformed by AI. Users would typically enter keywords into a search engine and sift through pages of results. AI streamlines this process by providing synthesized, conversational responses that aggregate information from various sources. In enterprise search, a particularly difficult problem, Glean is building an AI platform for knowledge discovery. In consumer search, this shift is evident in Microsoft's integration of ChatGPT into Bing and Google's development of Bard, both of which aim to offer more nuanced and complex answers to user queries. Another startup, Perplexity AI, generates accurate and context-rich content via chat interphase, challenging the current monopoly Google has on consumer search. The centralization of information through AI-powered search engines raises valid concerns about the concentration of power and the potential for users to bypass original content sources. In the consumer space this consolidation could also have significant implications for businesses that rely on traffic and ad revenue.
As we embrace innovative approaches to seeking information, legal considerations become increasingly apparent. The New York Times (NYT) lawsuit against OpenAI and Microsoft, filed in Federal District Court in Manhattan, alleges copyright infringement related to the use of millions of NYT articles to train AI technologies, including ChatGPT. This case has wider implications for the AI industry and content creators, raising questions about the use of copyrighted material in training AI models and the potential impact on traditional media outlets. It also highlights the need for clear guidelines and agreements regarding the use of copyrighted material in AI training data. The lawsuit is not an isolated case and it reflects a broader tension between traditional content creators and AI companies. Similar disputes have arisen between content creators and AI companies, signaling a growing legal and ethical challenge for the AI industry.
The rapid advancement of this field has been remarkable, evidenced by the growth and improvement of foundational models as well as the flourishing of new startups.
Applications of AI are unlocking new creative frontiers and augmenting software development in groundbreaking ways. In 2024, we can expect the dynamic interplay between open and closed ecosystems, the emergence of novel architectures that will shape the next generation of applications, and net new use cases and features. It will be exciting to see how these scientific advancements can translate into tangible benefits for humanity.
Further Reading: