aunhumano

I’m working on yet another AI Tutor, currently exploring synthetic students generation.

Context The coding ability of LLMs is growing continuously, and less and less code is being written by human SWEs (Software Engineers).

“Models constantly change and improve but evals persist” Look at the data No amount of evals will replace the need to look at the data, once you have a evals good coverage you’ll be able to decrease the time but it’ll be always a must to just look at the…

Why?

I’ve been eager for a long time to work on a project where LLMs negotiate with each other, to see if prompt changes can improve the negotiators.

A New Software Development Paradigm The increasing adoption of large language models (LLMs) in software development has brought significant changes, especially for teams integrating third-party LLMs via APIs.

Evals, or evaluations, are a set of tests used to measure prediction models performance on a specific task.

I’ve been thinking and doing some coding to make different AI’s negotiate with each other.

If you have or are building a software product with the intention of making it last over time, it would be wise to consider the constant evolution that AI is going to make in the coming months and years.

Evaluations before coding.

There’s been an explosion after the public launch of ChatGPT, followed by the leakage of the LLaMA weights and the open access to a OpenAI API.