The future of Programming will be AI first

Human readability is nice to have

Over at FRVR.ai our mission is to allow anyone to realize their dream game. Our aim is to build an integrated experience where anyone can direct GenAI to create any game - something we believe is already possible with current technology.

It’s a grand vision with major implications:

• Chat First: Non technical creators have no idea on how to use developer interfaces and tools, so everything must be accessible via a simple chat interface.

• Full Code: The concept of no code is fundamentally wrong. While the creators might not interact with source code at all, the AI must have full and unlimited access to generate any type of logic - not be constrained by the limitations of no-code platforms.¹

• Iterative: While it’s reasonable to create a picture from a prompt, it’s not reasonable to capture all the nuances of a computer game in a single prompt. So the platform must allow for iterative creation.

With those in mind, over the past months we have built a framework and scaffold that allows any LLMs to build and modify games by writing and patching source code dynamically. 

Initially our framework could be separated into four major components:

1. Context compiler: System that ingests all the context that exist for a given game, including game description, prompt history, game assets and current source code and pre-processes it into a spare data structure that is optimized to provide context for GenAIs to generate the output we want.

2. Prompt interpreter: System that interprets a user prompt and re-directs it to the GenAI integrator that can best solve the specific user request.

3. GenAI wrappers: Wrappers for different AI models ranging from LLMs to Image generators. Each with their own optimized configuration and context.

4. Integrators: A series of systems that validates, error corrects, post-processes and integrates the varied GenAI outputs into the current game project.

It quickly became clear that just optimizing this framework, while powerful, would not unlock the full potential of GenAI. We kept running into edge cases where significant prompt engineering was needed to prevent the AI from writing faulty code or hallucinating features that did not exist.

While these issues can mostly be solved by improving our integration framework and fine tuning models with data and feedback collected from creators, it's often simply faster, cheaper and easier to change our code structures and game engine to match the expectations and limitations of the LLMs. 

We quickly found that optimizing this new 5th layer of our framework easily led to the same level of improvements as the other four combined - leading to the major realization that the future of programming will be AI first.

Current generation programming languages, libraries and creator tools are made to be used by humans, supporting human strengths and compensating for human weaknesses, which often do not overlap with the strengths and weaknesses of GenAI models.

With modern programming languages there are many ways to arrive at the same result, with languages implementing a wide range of flexibility and shorthands such as inline conditionals and increment/decrement operators. 

As an example here are 12 different ways to loop and print the values of an array in JavaScript:

//1: For loop
for (let i = 0; i < array.length; i++) {
  console.log(array[i]);
}

//2: forEach
array.forEach(item => console.log(item));

//3: for...of
for (const item of array) {
  console.log(item);
}

//4: map
array.map(item => console.log(item));

//5: for...in 
for (const index in array) {
  console.log(array[index]);
}

//6: while loop
let i = 0; 
while (i < array.length) {
  console.log(array[i]); i++;
}

//7: do...while loop
let i = 0;
do {
  console.log(array[i]);
  i++;
} while (i < array.length);

//8: reduce
array.reduce((acc, item) => {
  console.log(item); return acc;
}, null);

//9: filter
array.filter(item => {
  console.log(item);
  return false; // so it doesn't actually filter out anything
});

//10: Array.from
Array.from(array, item => console.log(item));

//11: Spread operator with forEach
[...array].forEach(item => console.log(item));

//12: Recursion
function recursivePrint(arr, index = 0) {
  if (index < arr.length) {
    console.log(arr[index]);
    recursivePrint(arr, index + 1);
  }
}
recursivePrint(array);

This flexibility is great for humans who disagree on code style and implementation details. But to support this with GenAI you need significantly more data, increasing the training cost and model size while increasing the risk of generating code that fails to execute.

Counterintuitively this does not mean that the optimal language for GenAI is assembler code or some other coding abstraction with minimal functionality. Specifically:

• You want the AI to generate as little code as possible to decrease chances of it generating bad code.

• LLMs are trained on human language so a verbose API with easy to understand names perform better.

• To decrease costs you want to use as few tokens as possible, which means as few lines of code as possible.

What we need is a highly opinionated language² and SDK with minimal flexibility that has as many standard integrated features as possible. For now such a language does not exist, but we can approximate it by enforcing strict structure and formatting while having a huge standard library of functionality.

You should prepare now

I will be the first to admit that there are still many cases where AI struggles to keep up with human creativity and competence. However AI will keep improving at an exponential rate, something that is sadly unlikely for humans.

This, by extension, means that in the future it’s more likely that an AI agent will interact with the products and systems that are currently being built. Therefore: You should build for AI first, with human support and interfaces being a secondary priority.

You should build for AI first, with human interfaces being a secondary priority.

Creating human interfaces on top of AI appropriate infrastructure is trivial, while writing AI infrastructure on top of human interfaces is hard

In the future

It will take time to create and train AI on these next generation languages, so the first iterations of AI languages are likely going to be more opinionated versions of existing languages - this is certainly the path we are taking.

However as these languages evolve, the need for human readability will continuously decrease to a point where humans working directly with them is going to be as niche as writing assembler code.

It’s possible that the programming language step will be skipped altogether, with the AI dynamically generating experiences at runtime. Similar to how we don't teach AI to create images by using human tools such as Photoshop or Procreate, we teach it to generate pixels directly.

You could imagine a future where a game is simply a bundled deep learning model with a render pipeline. (I personally know people who are working on this)

Practical examples

Over at beta.frvr.ai 806 approved content creators, some with zero coding skills, have already created 3749 games using 140,605 prompts.³ Providing us with a sizable data set of things current generation fine-tuned large language models are bad at. Here are a few practical examples.

Scoping issues

The first version of FRVR AI games relied on a Game class existing and being initialized each time a game started. The intent was to have games code scope code separated from the rest of the games project. The structure of a game was approximately

//Games classes
class ExampleClass {
  //Class specific code
}
...
//Main class
class Game{
  //Games released code
}

The LLM models we work with, kept running into multiple issues with this structure

1. The LLMs would try to refer to variables existing inside Game class from other classes

2. The LLMs would try to fix such issues by writing global code which, a future that was disallowed.

To fix this we simply removed the Game class, instead putting game code in the global scope. Making the structure look like this:

//Games classes
class ExampleClass {
  //Class specific code
}
...
//Games released code

This is arguably worse code for humans, but the AI has no problem handing all code existing in the global scope.

Missing functionality

Our AI first game engine LK (Logik) is purposefully designed to be minimal, with the idea that it’s easier to add logic than it is to remove it. However it’s clear that we need to significantly expand the built-in functionality of this as quickly as possible.

A simple example is that the LLMs kept hallucinating an intersect function existing on a games object. E.g:

objectA.intersects(objectB) //AI expects a boolean to be returned

We had implemented:

LK.intersects(objectA, objectB) //Returns boolean

However as object.intersects is more descriptive, we changed our engine to only support this case.

Programmatically enforced code structure.

The LLMs write bad code all the time, but many issues can automatically be solved by adding a compiler and formatting steps to our pipeline. Here are a few examples of things that can be solved with this compiler step that would ordinarily have resulted in broken code:

1. Generating code in the wrong location. E.g. game code before classes.

2. Generating classes inside other classes.

3. Trying to change a variable that is been initialized with const

4. Re-using a class name

5. Trying to initialize a game twice

Additionally by having a compiler step gives us additional advantages such as:

1. We can rewrite code to allow for less flexibility. (This is mostly handled by showing and using the output of Babel.)

2. We can strongly enforce code structure

3. We can automatically detect assets and game objects using static analysis

4. We can validate API usage and arguments at compile time.

5. We can automatically rename variables that try to use reserved variable names.

Lastly, the compile step has allowed us to automatically convert games to the new structure when our interfaces change. Even major structural changes such as the scope change above, was transparently handled by this system.

Sadly this does not solve all issues. Luckily we also inject our own debug tokens, allowing us to direct the AI to automatically correct runtime errors.

Make your own games with AI!

If you want to make your own games with AI, you can sign up over at beta.frvr.ai/creator. If you put your username in a comment below, I will personally make sure you get approved.

Other articles in this series

• GenAI will change how games are played

1. Any no-code platform that is Turing complete is just programming by proxy using a worse interface.

2.  Some modern languages are also narrowing down the code flexibility as this complexity can also be confusing to humans. As an example Swift has deprecated ++ and -- operators

3. As of February 6, 2024