Artificial Intelligence and Machine Learning Research

Abstract Link to heading Let’s talk a bit about Model-Based Reinforcement Learning.

Abstract Link to heading A while ago we looked into Policy Gradient and Reinforce.

Abstract Link to heading Till now we have considered only learning the Value or Q function and estimating the policy from those.

Abstract Link to heading We already know how to learn the Value function, however we also know that the Value function by itself is not enough since it averages over all possible actions, instead of taking into consideration specific actions the agent sho…

Source Link to heading Paper link: https://openreview.net/pdf?

Abstract Link to heading In the last RL-Bite I wrote about Bellman’s Equations and the Value Function and now we will figure out how we actually apply these equations to compute the Value Function!

Source Link to heading Paper link: https://arxiv.org/abs/2405.14742 Source Code: https://github.com/JonathanGXu/HC-GAE Abstract Link to heading Graph Representation Learning is an essential topic in Graph ML, and it is all about compressing a whole Graph…

Value Based Reinforced Learning Link to heading In value based Reinforced Learning we learn a Value Function: $$ V_{\pi}(s) = E_{\pi}[G_0|s_0 = s] = E_{\pi}[\sum_{t=0}^T \gamma^t r_t|s_0 = s] $$ $G_t$ is the Total Return at time t, this is just the sum of…

Source Link to heading Paper link: https://arxiv.org/abs/2406.05391 Source Code: https://github.com/alipay/DUPLEX Abstract Link to heading I am a huge fan of Graph Machine Learning, it has a lot of cool applications, and I am particularly interested in So…

Explore vs Exploit Link to heading In reinforcement learning, we have an agent that has to take actions, for which it receives a reward.

Abstract Link to heading I am a huge fan of Graph Neural Networks (GNNs), and I am (a bit less) a fan of Large Language Models (LLMs), however they are hard to ignore.

Abstract Link to heading State space models are really close to my heart, I even have a dedicated page about them.

Abstract Link to heading In the last two years there has been a surge of Large Language Models.

Intro Link to heading Programming languages come and go and during my 20 years of coding I have used many of them to at least some degree (more than just hello world).

Abstract Link to heading It is notoriously expensive to train a Language Model from scratch, making independent research impossible and trying out new architectures extremely risky.

Abstract Link to heading Last time we looked into the weak points of State Space Models (Mamba, Mamba2), especially when compared with Attention-Based models (LLama, GPT-like).

Abstract Link to heading We have already looked into Mamba and Mamba2.

This series will cover a bunch of posts about State Space Models, their extensions and applications.

Abstract Link to heading State Space Models are awesome, models like Mamba and Mamba2 boast unparalleled performance especially when it comes to long sequences.

Abstract Link to heading This is not my first gig where I write about State Space Models.

Abstract Link to heading I have been working on a project of creating my own Large Language Model, as I am huge fan of T5, or to be more concrete I recognize the added value of having an Encoder-Decoder architecture.

Abstract Link to heading For a while now I have a new passion and that is binary reverse engineering and vulnerability exploitation.

Abstract Link to heading In a previous post, T5 the Old New Thing, we briefly touched upon CodeT5 and CodeT5+.

Here is a bunch of my posts about a super cool model called T5 its extensions and applications.

Why does T5 need a longer context?

Why T5 Link to heading A couple of weeks ago I run into the following paper Tiny Titans.

Lately, I’ve embarked on a side project: CodeBreakers.

High level overview Link to heading By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times.

The problem it solves Link to heading Imagine you have an irregullary sampled time series, where at various time points we perform interventions.

Probabilistic Matrix factorization is a simple but useful model for matrix imputation.