202505281637
Status: #idea
Tags: Project, Evolutionary Algorithms, Machine Learning, Aegis
State: #nascient

Aegis

Aegis is my end-to-end trading framework, a framework I have worked on for now two terms that involve evolutionary algorithms and Rust for the development of an extensible framework to make it so that users can select the metrics they want to optimize, select your assets, and then deploy the algorithm and then monitor it.

Right now, as I envision it, these are the blocks that I want to have:

• Ingestion block (take data from somewhere, likely a list of ticker symbols and then the model gets data online.)
• Strategy block: The goal of this block is to develop data-driven strategies based on grammatical evolution to automate the strategy discovery and keep them interpretable and (relatively) simple.
• Portfolio allocation block: This is the most advanced block currently, and it works in tandem with the strategy block to create an optimal allocation for a given strategy. Currently I use evolutionary algorithms as the main workhorse to test the allocations, but for some strategies it is perfectly possible that closed-form solutions might exist to find the optimal allocation.
• Meta-block: The block which combines the strategy and portfolio allocation blocks to make it more holistic. After all the optimal portfolio allocation is entirely dependent on the strategy, and the best strategies will vary based on what allocations are allowed. As a result, there should be some kind of feedback loop. I am planning to use a principle similar to the Expectation-Maximization Algorithm. So choose a default allocation (uniform) to generate strategies, pick the best k-strategies and evolve the best allocation for that based on our metrics, then use that allocation to evolve new strategies which hopefully will be even better than what we got on the first run. Do that some number of time. At a glance, I imagine that "convergence" wouldn't happen in the same sense, but if it did (in the best case) it would be in those simple cases where the optimal strategy under a given allocation is pretty much always the same. (I don't know how interesting those cases would be though.)
• Data block: Some block where all the data generated by the different blocks is dumped for displaying and analysis. Cool uses for AI exist here, maybe to help think of new research directions.
• Deployment block: The part responsible for the shipping to an actual API and making trades.

As I see it, the most fundamental blocks are the data ingesting, strategy, and deployment blocks are the most important. And like by far, pretty much everything else is refinement on top of that. So frankly, it makes a lot of sense to keep those worries for when these fundamental blocks will be completed and tested.

Overview

This is subject to tremendous change.
The goal is to have a data-driven trading framework able to discover, test, and deploy strategies automatically.

The general architecture is as follows.

1. The ingestion layer would take a list of tokens (these are the financial instruments to get instruments on)
2. As much data as possible is pulled from the APIs to train/compute an sampler which will be used for the strategy generation.
3. We will use grammatical evolution to generate strategies with respect to this data using a default uniform allocation.
4. Best strategies are sent to deployment block for proper backtesting
5. If greenlit, the strategy is deployed in paper trading for a while to gather more data
6. Finally if it passes, we properly deploy it.

After a strategy has been running for some time, and has proven itself worth the hassle, doing the further optimization of allocation is then more logical.

For now, getting it done is a matter of implementing this MVP. I am writing this after an hiatus I took to learn a lil bit more about javascript and frontend design. Now I feel like I can actually get into it.

So I got two main plans.

1. My priority is: Get the working trading system MVP with the data ingestion, grammatical evolution, deployment pipeline complete and working. I want to have something that is usable before the end of summer.
2. Do a thorough code-review along with explanations so that I can go and explain the how and why of everything, along with what I would change.

Since the priority is the MVP, let us start talking about that.

1. Getting a Trading framework of the ground

Ok, so the goal is to use grammatical evolution to evolve strategies. I want to use Rust, but that is besides the point. The first concern is really just creating a good grammar. Grammar quality is key to ensure good exploration of the search space and solutions that are high quality.

This is why, our first stopover will be here: Developing a Grammar for Aegis, Money speaks!

Relevant Links