learning-epistasis-principles

Such finding strikes me as extremely unintuitive as I’d expect to see an extension of complexity and capability in gene expression as one grows the interconnectedness of a system. I would also expect to just have highly interconnected systems because nature is messy and robust.

Of course I believe in mashed potatoes and useless entropy. However! The chaotic landscape and my insistence in the complexity and widespread intelligence of life prickled my intuition to think of epiplexity.

With this work I hope to develop an information-theoretic explanation on why these landscapes are learnable and what are their limits.

My approach intends to unravel underlying characteristics of the landscape that might be driving epistasis. The benefits of finding such traits are: transferability, chemistry and physics constrain what genes can do, that means understanding these rules might give us a general framework to understand the process of genetic epistasis, and sample efficiency, if a small number of underlying properties generate the interactions, it will take far fewer measurements to pin a landscape down.

In order to measure the capacity of expressing information I’ve used epiplexity. Epiplexity measures how much learnable structure something has, from the point of view of someone with limited time and data. It splits the information in the part you can actually learn and reuse, and the part that just stays unpredictable to you. The neat thing is that “random” becomes relative to the observer. For example, even though a pseudorandom number generator comes from a tiny program it looks like pure noise to you if you don't have the generating key, that means it will have a near zero score in epiplexity. One measures it by training a model and seeing how much it had to absorb [TODO] before it stopped being surprised.

I devised some really simple experiments to measure what’s the capacity that systems that are highly interconnected have to express information.

Initial explorations

I. Synthetic genomes.
Using 64 genes, I’ve calculated epiplexity for genes without interactions, every pair interacting, each gene depending on 2 neighbours and each gene depending on all 63 others. We also looked at results with fitness assigned at random.

II. Controls.
Random landscapes and pure noise must read zero, additive landscapes must read high. (An additive landscape is one where every position contributes independently, each amino acid adds or subtracts a fixed amount, and the total is just the sum. No interaction anywhere).
This was reproduced reliably.

III. Looking at epiplexity in a GB1.












GB1 is a small bacterial protein, (a protein is a chain of amino acids that folds into a given shape, the shape is responsible for the success of a protein’s task). For this given protein its job is to bind to an anti-body, so how well does it do that is our fitness measurement.
Each position can hold any of 20 amino acids, which means 20⁴ = 160,000 combinations.
These four positions were chosen because they interact unusually strongly with each other.

For this experiment we want to build synthetic twins of GB1 so we can isolate the different traits that might be influencing its ability to be understood, or learned.

-The pairwise twin only ever lets two positions interact at a time. It asks: what would GB1 look like if its interactions were all simple? Answer: it would look more learnable than the actual protein.

-The spectrum-matched twin keeps the same amount of interaction as GB1 but shuffles which amino acid combinations get it. So if in the real protein a particular pairing of two amino acids mattered a lot, in the twin that same large effect gets handed to some other pairing instead, picked at random. Answer: the arrangement matters; the real protein has more epiplexity.

-The random twin assigns a fitness to each variant independently, with no structure at all. It has zero epiplexity.

From these experiments we learn that GB1 contains a structure. However, it also contains a lot of random noise, the reason for that is that about 36% of its variance is in interactions among three and four positions at once, and at this size that’s effectively unlearnable (four positions interacting jointly has roughly as many independent cases as the entire landscape has variants, so there’s nothing to generalize from).

IV. The shuffle test.

Keep every interaction amount exactly as measured in the real protein, and shuffle only which amino-acid combinations carry them. Anything that we can find is explained by the amount of interaction alone. Anything that disappears must have depended on which specific combinations carried it.

When I shuffled, most of the learnable structure went away. Which is actually the coolest result, in my opinion. At 16,000 measurements the real protein had about 2.7 times as much learnable structure as its shuffled version. So roughly 63% of it was coming from which amino acid combinations carried the interaction, not from how much interaction there was in total.

That rules out a simple explanation of GB1. You cannot describe this protein just by saying “it has this much pair interaction and this much three-way interaction”. Two landscapes can have exactly those same amounts and one of them is much easier to learn.

Something about which specific combinations matter is itself a pattern, my best guess is because real amino acids have real chemistry and are subject to real physics, and the ones that interact strongly do so for physical reasons rather than at random.

Future explorations

1. Is chemistry/physics the explanation? How to create environments in which we can test this?

2. We can try different topological structures for how genes are connected. Does it matter for the ability to something to express information if it's random or a ring, or some other kind of connection.

3. Is evolution doing the best thing? Is it maybe doing the best thing if all variables are considered. Similarly to the bubble sort algorithm paper by Levin and others?