3 songs in 1 day

 a good day for robert hunter! 

ripple

brokedown palace

to lay me down

before and after

 what we knew before in great detail about the EU benefits and on balance...

and 

what we learned after we were told there were no downsides to leaving the EU 

without much mitigation

shocking really.

AI will boost the economy by [insert your ludicrous claim here]

 Every day there's another request from policy people to tech people to predict how and where AI will help the economy. The latest one I've seen in the UK had a "pulled out of thin air" claim that AI was going to boost the economy by 6%, and represented the 4th (or is it 3rd or 5th) industrial revolution since {fire, wheel, steel, steam, production lines, electricity,advertising,internet, etc etc)

Aside from the fact that most the past revolutions didn't have the impact claimed, or certainly not in the way described e.g. miners and factory workers were poorer than tenant farmers, both in money terms and in quality and length of life. And they were a large fraction of the population for a while. I bet the same will be found about office and information workers too. this isn't just about bullshit jobs, it is worse - it is about the cheapest way for capitalism to operate.

so for AI to do the things the techno-optimist brainwashed dimwit policy people claim, it would largely be working in optimisation of existing practices. since you can't live in an AI instead of a house, nor can you eat or breath an AI, or ride the AI across the land and sea, nor can you power your heating or A/C with an AI.  All it can do is make some of these things work a bit better.

But we've been optimising systems for hundreds of years - partly through the "invisible hand", but also explicitly - transportation and  utility providers (often when state operated) employed hundreds of operations researchers to optimise the services. The electricity grids and rail networks were marvels of efficiency. They could only get worse by privatisation.

So now we expect private companies operating AI to make sub-optimal services better? ironic eh. Absolutely no way, jose, is that going to work.

Just another HS2 waiting to happen, just another NHS IT project hovering in the wings. No-one honest should be involved in any of these projects until someone (aristotle, jesus, laozi, marx, kropotkin, piketty, someone even smarter?) devises a replacement for the economic fiasco that the descendants of Adam Smith and F.A.Hayek mysteriously still adore.

Hey, maybe the sign of real AGI will be that a machine will propose an explainable, feasible, and deployable replacement for capitalism and the patriarchy.

We live in hope. foolishly.

Fermi's last paradox.

Some people have been driven to revisit Fermi's paradox once more due to the recent revelations about UFO sightings from the USA.

Let's revisit the basic idea - we have some notion of the number of galaxies in the universe, the number of stars in galaxies, the distribution of stars with exo-planets, the probability those exo-planets are in the goldilocks zone, and so some idea based in 1 prior (life on earth) that intelligent beings must be out there, and, indeed, out there in pretty large numbers.

This is an sample bias of massive proportions.

While the idea that life should be likely given the right circumstances, and the observed statistics of the basic ingredients (at the planetary level) support this, the Fermi paradox is that, given how long we have been around as an intelligent species, capable of observing stuff like this, then the chances that others are around, at the same time, should be very high, and yet we have not (convincingly) encountered any of them, or at least not visibly, nor have their deigned to contact us.

So I have two arguments about this not being a paradox at all, one based on science fiction, the other based on fantasy.

1. SF - Asimov's Foundation series started as a riff on Gibbon's Decline and Fall of the Roman Empire. In it, he posits a galaxy wide empire which will eventually collapse due to internal contradictions (actually, he'd never have used such an overtly Marxist term, since he was an ardent capitalist, but his psychohistory theory smacks of marxist dialectical materialism). In this, the only way the "dark ages" that follow an empire's collapse, could be mitigated was via the Foundation, and even that couldn't solve noise introduced by accumulated small deviations from individual behaviours, so needed a Second Foundation to correct the noise (even they nearly lost due to the Mule, an extreme perturbation). So good old Prof Isaac basically invoked magic (ok, so in the spirit of Arthur C. Clark, it was dressed up as a technology so advanced as to be indistinguishable from magic). Indeed, in later books, he merged two of his future histories together from the robots and the empire, he made up yet more magic (Gaia, and Emergent Ethical self-modifying sentient artificial beings) ... sigh

The key point here is that Empires don't last. We have a set of past "civilisations" we could use to model the distribution of life times of organised societies - Mesopotamia, Songhai, Mayans, aforesaid Romans, the Brits, the EU, the USSR, etc etc - look, they don't last long. 

Worse, when they encounter another empire (British v. Moghul), inevitably one disappears.

So in space, chances are most organised tech societies don't last long enough to find one-another, but occasionally, when they do, one absorbs the other. Only it still fades away after (say) 100 years.

And note if FTL is not possible, we need to sustain a tech society across generational star ship lifetimes, which could be tens of centuries at least.

2. Fantasy. Maybe dark matter and dark energy are anathema to intelligence. Maybe there is "dark intelligence" which absorbs all smart being trying to make it through to the next light zone. Maybe there is some sort of truth behind Good Omens and other stories - there's a bunch of adversaries out there, but they aren't what we would call "civilised" - they are creatures from hell. After all, didn't a very smart human once say "Hell is other people". Maybe Fermi's paradox isn't a paradox, it is just that between all the little possible utopias is a vast abyss, not full of nothing, or zero point energy, but full of demons.

Note Frank Zappa already remarked that Hydrogen is not the most abundant substance in the Universe, and that stupidity was far more abundant. Hence, Dark Matter, Dark Energy, Dark Intelligence.

larging it language models

 so lets deconstruct this bullshit.

generative models - i wrote a random number generator in 1976. it used an ancient technique from the past still in use today coz it works - it was generative - that doesn't mean anything. 

foundation ai - what's that about? the mule defeated the foundation, last I heard. what a load of tosh.

large language models - not at all - statistics of utterances, unutterable bollocks. that's not language, that's verbal Diarrhoea.

"attention is all you need"? sure, if you have nothing to tell people about, it sure is.

interestingly (since penning this rant) I've read a whole slew of papers about shrinking NNs in general, and pruning LLMs specifically...still a research thing, but the Lottery Hypothesis suggests for now, this is no longer just post-training (just starting from a smaller architecture produces lower accuracy models...hmmm why?)

phew. lets get shot of this hype cycle and back to fixing the planet.

The Institute for the Musing on Dangerous Ideas

 The Institute for the Musing on Dangerous Ideas (IMDI) has received a great deal of money over the past 15 decades from the ex heads of state from several advanced civilisations. The accumulated wisdom and experience  of these grey hairs is, of course, not in doubt, and is, of course, most welcome.

However, it has become clear that they completely failed to predict the massive global success of democracy, and as such, have demonstrated very poor value for money. The rapid rise of generative politics, whereby large populations of people engage in decision making has caught despots and demogogues, populists and popes, all by surprise. "Why weren't we warned?", we hear them cry.

We propose a new Institute, known hereafter as the Office for Responsible Brains (The Orb), which will concentrate minds on the problems at hand. At the same time, we propose that The Orb be funded at least at one order of magnitude more than the IMDI was, to avoid starvation of resources, (at least amongst the Orb members. Of course, competing organisations, like Norway and Iceland may struggle in the face of such blinding enlightenment that we expect the Orb to bring to bear.

Never again will former great leaders be caught with their pants down.

P ("paddy") Ashen-Face

T ("tone") Def

A ("B") de Kerfufflle

Acknowledges to the following for tech support and advice

P Thrall

E Mask

M Hucksterberry

probablistic programming and protocols

 for a long time we've known that a number of processes in the internet are heavy tailed - the distribution of files (web page/video)  is typicalyl zipfian, so transfer times are heavy tailed, and also protocols like TCP induce burstiness from a number of different root causes (timers, AIMD, interface packet scheduling, stat muxing of many flows of different duration and round-trip-time etc etc) 

so people have written analysis tools that capture these stats (e.g. to do measurement based admission control for traffic that cannot simply be described by a mean or peak rate) - which can be quite parsimonious....they've also written generator tools so that synthetic packet traces can be built for simulation or testing - typically some sort of fractional brownian thing - one debate not to get drawn in to is whether "TCP is fractal" - it does't really matter - you just want. a time series that has the requisite self similarity (e.g. right Hurst parameter) so you can dimension buffers for some delay stats for traffic flows...

some very clever maths was done to reduce all that to some simple tools would do 99% of what you want - e.g. see papers from the Measure project.

this was all reminded to me yesterday in a talk about stochastic processes (and cheaper ways to do gaussian processes) for probablistic programming. The way the speakers tackled it was using a variational inferencing approach to split the gaussian into components (standard autoencoder hack - simplify, then synthesize) - the problem with this is that, while it works well (see \pi-VAE for details), it is hard to explain, or more importantly, interpret actually what component stochastic processes are being put together to get the fit!

two ways to tackle this (in my view) - one would be a \phi-ML approach, as above, where you have an explanation, but it is hard to solve, but you can approximate it directly.

The other idea came up in a question from an attendee, which was what about using neural processes, which are in some sense, a basis set of functions (think, like Fourier or Laplace transform) - this is directly speaking to what one is doing (trying to build a particular guassian process) but has unfortunately high costs in the general case, but works well in a lot of other classes of physics problem spaces.

I don't know enough about the stats to fix the neural process approach, although using an XAI technique on the VAE might yield an affordable interpretation of that approach, but I offer the observation about long tailed network traffic, which looked pretty intractable until some folks thought of some simple tricks....and maybe mixing the physics ML neural process with those tricks might yield an idea for how to make the neural process for stochastic behaviours efficient for some (very) common cases?

So what might make a suitable basis set of functions for spatial data ? well how about a poisson point process? so a super-position of a set of poisson point processes with different \lamdas would likely yield almost anything you want - main challenge is to decide how many (different) ones suffice to get a good match