When Chat-GPT came out a few months back, coders were losing their minds at the “capabilities”. All of a sudden the utopic dream of having AI do code for us excited and then terrified us.
I’m a believer in AI, and I know for a fact in the coming years it will get more and more capable, and one day it will take my job. But it’s not today, and in this article, I want to make it clear why it’s not today, and how you can survive tomorrow by preparing today.
A smart horse runs when it sees the shadow of a whip.
Chat-GPT has given us a glimpse of the future of work, without significantly changing the present. It’s the shadow of the whip, and the whip will hit hard in the future if you’re not ready.
But let me be clear, It’s not today, And it’s getting worse tomorrow.
Why it’s not today
1. A limited handle on real-world Complexity Ask the AI to write a piece code to “sort a list”, and it will do great. Ask it to “write a C# method to upload an xslb document using a library that supports xslb in .net core” and it’s going to be lost, horribly.
It may create a code, but it’ll use a weird library for reading xslb files that you don’t want to use.
Let’s get real, writing code for relatively common use cases is great, but where the devs really get lost is in the nooks and crannies of software requirements.
Can it handle it? — No!
2. Can’t keep up with the Kardashians Technology is moving fast. New Libraries, APIs, and services are being developed every day. Software versions keep updating. Let’s take the case of .NET and C#. Chat GPT was trained on code samples from perhaps the era of DotNet Core 3–4 and C# version 5–6.
We’re currently at DotNet Core version 8 and C# is at version 11. Can it write code with the new syntax and patterns of DotNet? — No!
3. Limited Context window Sure, you can create atomic functions or classes or files. But what about writing code in a module that has a ton of surrounding context? Let's say we need to create a new service in our Sales Module for our ERP application. Well, how do we supply all the models, contracts, sibling services, and data access layers so that the code that is outputted by the LLM actually can be used in our code-base?
Can we actually supply all the “context” that our code needs in 4K tokens? — — Very rarely!
4. Multi-File cases Modern applications are not written in a single file, we need to modify multiple files in different directories to achieve the desired integration.
That means the LLM needs to: 1. Create new files in a particular directory structure 2. Update old files in certain line locations in a particular directory structure.
This is where LLMs struggle as they are not aware of the file and project structures, and mess up horribly.
5. Style guides No one size fits all, and each org has its own conventions. We like our private variables to be written as “_privateVariable” and some might prefer them as “privateVariable”. Method naming, file naming, and project structure are all nuanced, and no amount of prompting can encompass the “requirement” as well as the “style”, in a single prompt.
Is all hope lost?
Humans are amazing. People are working on these problems (including us at Core Maitri) and we will find a solution.
Here are some promising candidates.
For longer tokens and larger prompts:
LongNet: Scaling Transformers to 1,000,000,000 Tokens — https://huggingface.co/papers/2307.02486
LTM-1: LTM-1: an LLM with a 5,000,000 token context window — https://magic.dev/blog/ltm-1
For existing projects and multi-file support
Auto-GPT: https://github.com/Significant-Gravitas/Auto-GPT
Meta-GPT: https://github.com/geekan/MetaGPT
We’ll get there, but it’s not today, and probably not tomorrow
Core Maitri is an enterprise software consultancy specializing in Excel-to-Web, AI Integration, and Enterprise Application Development services. Our approach is deeply consultative, rooted in understanding problems at their core and then validating our solutions through iterative feedback.