GenAI in SDLC

created: Fri, 03 Jan 2025 17:23:34 GMT

Notes and reflections on applying LLMs to Software Development Life Cycle.

RAG

Premises:

  • Providing LLMs with context seems useful to improve generation accuracy
  • Context is limited by the model context window, also there are concerns about overloading the model with data
  • Vector databases and Embedding models are used to build context, different embeddings provide different results
  • Chunking, indexing and re-ranking is important to build more accurate contexts

Embeddings and re-ranking as a service:

Links:

Reflections

  • The idea that sequential information retrieval (aka chat) somehow works better for humans.
  • Oftentimes bad quality code that "does the job" is good enough.
  • The most helpful as a side window with advices on context.
  • The model should be able to generate multiple ideas and explore them.

Quotes

The solution is probably integration of LLMs as a part of a composite system that includes database storage, a code execution environment, and multiple agents to form a goal directed posit - evaluate loop.

I think we expect AGI to be much smarter than the average Joe, and free of occasional stupidity. What we’ve got is an 85IQ generalist with unreliable savant capabilities, that can also talk to a million people at the same time without getting distracted. I don’t see how that isn’t absolutely a fundamental shift in capability.

I think this depends heavily on what type of coding your doing. The more your job could be replaced by copy/pasting from Stack Overflow, the more useful you find coding assistants.

As an "older dev" who doesn't like AI, the thing that annoys me most is the UX is horrible. It's like arguing in chat with an extremely overconfident junior dev who isn't capable of learning or improving with time and experience. That's just a miserable way to spend time. I'd rather spend that time thinking clearly about the problem, and then writing down the solution (clearly).

Links