Learning Generative AI (GenAI) Development in Public
Join me on an unusual learning adventure as I modernize my software application development skills with natural language and LLMs
Before I dive in, there are a few things going on with this Substack to bring you up-to-date on:
I just started this “Learning Generative AI (GenAI)“ writing project recently. I’m a longtime mobile and web developer expanding my skillset into the GenAI space. I’m still figuring out the Substack community, ecosystem, and the Generative (GenAI) tech content that is already flowing in this space. I had avoided Substack for years, but now that I am here, I don’t know why. Lots of great content and contributors in this space. I already like it much more than Medium and other swipe swindling, gaze gluing, scroll holing apps (aka social media).
I’m still dusting off my writing skills and starting to use GenAI as a writing tool. I don’t need it to tell me what to write, so it is a writing assistant, editor and sounding board for me so far. I also have not tried to define my audience as of yet, but I’ll stick to working on writing habit for now.
I was sharing my process for evolving a new tech stack for building LLM-backed apps. I wasn't building a completely new stack, but instead relying on my trusted methods for selecting the layers of the stack I planned to use for full-stack GenAI app development. Unfortunately, what had once been tried-and-true turned out to be tired-and-blue in this rapidly evolving world of software development that leverages natural language at multiple layers.
You Are Here
Let’s get oriented on what I’m writing about.
My first post laid out the layers of the stack I was considering. The second post started to offer some details as to the what and why of each layer, as well as sharing some of my initial implementation investigations.
In my first stack layer exploration with a new CMS, I quickly ran into the typical “not on my machine“ configuration issues that almost always come up whenever diving into new stacks or frameworks. That was expected. But what I quickly realized after spending more time that I wanted on the CMS was that I was starting this GenAI app development learning adventure backwards. I was not working at the highest level of abstraction for my given objective. So that begged the question: What is my primary business objective?
The primary objective for this learning adventure is to learn how to build LLM-backed applications and services. Does learning a new CMS or web framework get me there? Not directly. Those can come into play down the road. For now I needed a much faster way to prototype some basic workflows or spin up endpoints that execute some GenAI processing, be it text or multimodal.
That brings us a little closer to where I am today. I did have one more stop to make on my way to where I am today though. That was to kick the tires on no-code / low-code visual AI workflow automation tools. Surely these types of tools will limit the time spent on messing with configuration and cranking out boiler plate code. This would bring me much closer to my objective as I need to be to get moving faster…or so I thought.
New and Shiny Visual AI Workflow Builders
I dove right into n8n, a partially open-sourced visual AI workflow building tool, to explore an idea I’ve been working on for a while. The first step was to create a flow that could directly ingest and parse incoming emails in their raw format. Using n8n, I quickly set up an endpoint to which I forwarded raw emails from SendGrid. I was able to quickly test my flow and watch as the email was successfully received and parsed by a node I had just visually configured. Great!
Next, I started to work on using an LLM to extract entities and summarize the content of the email. Adding credentials for my OpenAI API account was simple enough. Firing up and configuring an “Information Extractor“ node was easy as well. I sent another test email, and there it was. My extracted entities and summarization as a JSON Schema formatted output. Bravo!
Wait…it made up some names. OK. I swapped the LLM I was using via the node configuration settings and I changed the temperature. Next I tweaked the output format a bit. Hmmm. Totally different response quality. Testing it again tripped the rate limit for the model I have selected.
Ohh… Wait a minute. I see the problem.
One more level of abstraction
The shortest path to learning about LLM app development is by have as limited a set of intermediaries between me and the LLM to get started. Not only that, but I need a way to keep track of what LLM’s are out there, what they are good at, what their token, context windows, pricing and API rate limits are. And that is just to get started. Are they open source, closed source, open weighted and suited to my app’s business objective? There will be no LLM apps if I can’t tackle how to manage all this information about LLMs themselves.
With the rate of LLM development and releases, this is a significant issue to tackle. When you’d adopt and learn a new framework in the past, you could count on having almost a year to work with it before you had to learn the new major developments with it. LLMs and the tooling and orchestration layers evolving around them are moving so fast, as an application engineer, how are you supposed to keep up?
Managing information overload
Effectively managing all this information around LLMs is essential for developing LLM-based applications. Without staying informed about the rapidly evolving landscape of models and their capabilities, you will continually struggle to choose the best option for your application or fail to maintain up-to-date knowledge of LLM trends.
That is where I’m at today. I’m working on a few approaches to these issues from both the hands on standpoint interacting with LLMs as well as the knowledge management aspect. I’ll come back to those in future posts.
If you’re a software engineer or application developer like I am and want join my learning journey, subscribe to stay in the loop.
If you are someone with experience in the GenAI application development space and have anything to help illuminate the various waypoints on this journey, your input will be greatly appreciated.