Friday, September 20, 2024

weaponising the supply chain - thanks but no thanks, mossad

By intermediating the supply chain, Mossad appears to have been able to subvert safety in various mobile devices (so far, pagers, walkie talkies, possibly some phones) - one speculation is that they undermined the current limiter or other safety fature that stop the battery overheating and catching fire - and put in some interface to allow software (e.g. via specific messages to the device) to trigger this behaviour - rather than, say, just putting a few grams of semtex in the deveice and turning it into a small IED.

Because this was done at scale, and somewhat scattershot, the normal trust in the safety of devices bought through regular supply chains has been undermined. Imagine if Mossad had decided to do this via several made up intermediaries who sold through the Amazon Market place for example, especially with relatively low value items that aren't typically checked when being shipped internationally. Great. Now the idea is out of the box, it is another extreme example of asymmetric warfare - lots of organisations could re-implement it easily[*]. 

This instantly, a lot of organisations now have to worry about people who, having innocently bought such a device, (or indeed bought one off of someone else who got one of these exploted gadgets) want to travel

We currently ban e-bikes on trains in the UK because, even without state-sponsored terrorism, safety features on e-bikes are not terribly well checked (they don't have the equivalent of a regular MoT/roadworthiness/emissions check, which might help a bit). We also don't allow really large capacity power banks on planes. In these cases, the risks are higher even if the occurance of fire/explosion is rare, because the energy in the device is so much more.  Nevertheless, the explosions seen in Lebanon would be extremely dangerous on a plane, whether in the passenger cabin or the hold. 

Or we'll only allow devices where the battery can be removed and kept seperate from any trigger circuit. (useful for consumers who want to replace old, knackered batteries too).

So maybe we will see a ban on carrying any mobile/rechargeable device on planes for a while, until some certification (including tamper proof sealing of post-certified devices) is available.

Of course, Mossad will then subvert the certification labs next, no doubt.

Just to start with I think we need to stop people with Israeli passports traveling outside of their country as they represent a clear and present danger to everyone in the world, not just to innocent bystanders in market places in lebanon (or gaza). Until they can assure us that they are not going behave so irresponsibly, and regain any possible level of trust they might once have enjoyed. Of course, most their agents will also have other passports too.

Update - this Bunniestudios blog is a very useful detailed analysis of the howto....

* footnote - why the west didn't launch cyber attacks on Russia's infrastructure (e.g. taking down all their power and comms) when they invaded Ukraine was a) revealing the tools the west has and b) inviting a retaliation which would also have succeeded, were both v. bad ideas. Mossad has just revealed that it has no clue about this type of precautionary principle. Well done. guys.

Thursday, September 12, 2024

explainability, next to reversabilty?

 XAI has many flavours (includnig interpretability as well as explainability) - au fond, the idea is to shine a light into the black box, and not just say why an input produced an output, but potentially show the workings, and, in the process, quantify uncertainty in the output (confidence)- in the process of using an AI that does produce these outputs, the user can necessarily gradually construct a model of what the AI is doing (and why, given the user knows the inputs too) Hence, in a sense, this is like debugging the AI, or indeed, modelling the AI. i.e. reproducing the AI's model. In the end, the user will have reverse engineered the AI.  This is an indirect, amd possibly time consuming way of reproducing the model, effectively if not actually. Ironically, in some cases, we may end up with a more accurate, or a cheaper model, or both. 


Of course, you may dispute that the model we learn is not actually the same as the thing inside the black box - the analogy of boxes and lights is, of course, nonsense. If we were to know the actual machine learning model (linear regression, random forest, convolutional neueal net, bayesian inferencer etc, and the actual weights (model parameters etc) then it wouldn't be a black box, and we'd be able to simply copy it.  various techniques can be used even for quite complex machines, to relate the model parameters (e.g. CNN weights and clustering) to the features the model is able to detect or predict. This is the direct approach. In this approach, we are also able, potentially, to simplify the actual model, removing components that serve no useful purpose ("junk dna"?).

Either way, any sufficiently advanced and thorough explanation of an AI is going to be a copy.

I wonder if the world of LLMs is resistant to XAI techniques partly (honestly) because very large models would be very expensive to re-model these ways, but also partly because some of the proponents of GenAI technlogies like to retain the mystery -- "it's magic", or perhaps less cynically "it's commerical in confidence". 

However, if we want to depend on an AI technology for (say) safety critical activities, I think it better be fully explainable. And that means it will be transparent, actually open, and reversable (in the reverse engineering sense). 

Monday, September 02, 2024

what if cat species were named after greek food?

mossaka, the mouser

calamari, the cat of nine lives and nine tales

tsigarides, the top cat

kleftiko, the clever cat

stifado, the sedentary cat

marathopitakia, the mischief maker

dolmadakia, sleeps all day

add yours here...



Monday, July 15, 2024

Unsustainable? Inconceivable!

 It seems that people are just starting to cotton on to the fact that the new wave of giant AI is not very sustainable. Mostly they look at the cost (communications, storage, computation, electricity, water) of training. 

But there's another couple of  costs being ignored

1. it has taken 32 years (give or take) of the WWW to get to where we have all the material avaialble today, including millions of websites, blogs, scientific and other academic open access materials and wikipedia and so on, as well as huge numbers of photos, songs etc - this represents a massive investment by 100s millions of people over more than a generation.

2. really useful data out there has been curated (a.k.a. wrangled) so that it doesn't have too many lacunae or errors, and may be statistically representative - it may also be accompanied by meta data (describing its meaning, but perhaps also labelling features in the data with meaningful tags - especially useful, for example, in medical images or satellite images of earth, but also just simple stuff like names of people in pictures, and GPS/location data of a photo or movie. This also took both time&effort, but also expertise - humans spent a while using their knowledge, and possibly skills, to add that extra information.

Of course, a special class of data is code - and open source repositories have a lot of that, associated with meta data ("documentation") and labelled (e.g. with commit logs describing bug fixes or features added, by whom, and when)

While they may offer all this data for "free", using it to train an AI is being undertaken lightly as if this is the same as using an image or a blog or a piece of music for entertainment or education.  

By absorbing this mass of material into a model, what is really being done is absorbing the prior information that gives more than a slight hint about the model that was in the minds of the users who created the original content. That is to say, their labour is being appropriated, not just the fruits of their labour.

So if you want another common-crawl's worth of data, be aware a lot of people will quite like to be paid for their effort next time around. And can you afford a payroll with 100M expert employees working for 32 years? Really?

Wednesday, June 12, 2024

career pathways at the Turing - some ideas

 


Turing v. University v. Industry Lab v. Government


Academic classic career path


school (work experience)

undergrad (remember urop/intern) ->

masters (1-3 years)

phd (internships 2) 

post doc contract 

research fellowship (RS/RAEng, UKRI, Leverhulme etc etc 3)

faculty position (assoc prof->asst prof -> full prof (tenure)


About 30% of our students -> Masters, and 50% Masters -> PhD

Some take 1-2 years between each step , out in industry (finance or tech)

About 20-30% students switch to another discipline (CS -> finance GS/Accenture/E&Y etc)

About 30% exit at any stage to go to compute tech industry (1) including from Academic

(typical Cambridge non academic destined CS PhD CS might go to  Amazon, Microsoft, Google, Apple, etc, but also startups

Some people move from study or research to government roles (civil service or policy).


Turing 2.0

Turing 1.0 for years 1-7 approximated most of these phases with three obvious differences. (note in non AI/CS area, we have other models - e.g. CERN, Sanger, Welcome etc etc)

1. RSDE (REG) was modelled on jobs for senior software engineers at Microsoft Research, and has created an entirely new (but non industry) based career pathway (and is being adopted across UK and more widely

2. Internships (we have incoming in the Turing, as enrichment students, but not outward

3. Fellowships - here's where a nice transition from being supervised/trained in research, to writing own funding proposal to get often 5 years, of autonomous funds - uni/academic departments love it, as they get

a person who is REFable, does leading research (the funds go to people who write the very best proposals, and in some cases, e.g. ERC have to have already a cool very high publication profile already

The funds often only require someone committed 50% to research with up to 50% free to teach (or do other jobs in the uni)

The process of getting UKRI/RS type fellowships is a high quality control mechanism for some people to gain research leadership skills -some people even do several, and they can be got right up to senior level by academics.

It is the way one then learns how to write, and run larger research proposals / projects, with more junior  roles (PhDs, postdocs) for the fellow to manage to get a team to tackle larger scale problems, possibly  leading to even larger, collaborative grants (programmes of work) with other institutions (going further, possibly, adding partners from other disciplines.

Fellowships reduce, but do not eliminate the precarity of post-doctoral contract research jobs...the Turing could consider a similar role.

Moving from early career to leader

At some point in this career path, someone may get higher levels of recognition and be invited/elected 

to various scholarly bodies, to work on advice to the funding agencies, and to government departments.

And of course funds can be got from industry or philanthropic sources, but need some level of autonomy and visibility for someone to be seen as eligible (unless they move with the money to somewhere else where they would be come eligible on arriving...ERC grants sometimes have that effect).


Something not mentioned in this so far is spinout or startups. Policy/careers for this are highly variable across the UK (and EU and US) today...leaving an institution, then coming back can be done (Stanford in US and Cambridge in UK allow it - it is more common in some sectors, e.g. pharma, than others).


Being an academic almost always involves teaching duties - the major plus point of having students in 

the supply of smart, educated known quantities for the next stage of their career. Another fine point 

is that student projects are niece ways to explore ideas (especially at masters level) - the nearest the 

Turing has to this is enrichment students and the University partnership. 

However, the longitudinal relation between student and advisor is outside the Turing.


Another good thing the Turing does involving groups of people including students is data study groups.


And what support does the Turing give for these various notions of career pathway development?


Anyone should be able to aspire to one day being an S&I director, or Chief Scientific Officer




Recommends:-

Have a clear pathway for career progression, all the way to the highest level, not excluding S&I leadership, chief science etc.

Celebrate (and re-enforce) the REG/RSE model.

Deal with precarity in a way at least as good or better than current University practice.

Consider how Turing employees can be empowered to seek external funding for (e.g.) fellowships if they are planning to transition to academic posts later.

Similarly, enable and support employees wishing to spin out work and engage in startups. In that same breath, support their return from startups in appropriate ways (tech transfer achieved etc).

Consider supervisory roles and leading training tasks for the above

For career pathways with intention to transition to industry, look at senior possibilities for people liaising with such partners while at the Turing.

The Turing has strong relations with some government departments (e.g. defense&security, but also health, transport, climate etc), and so career pathways that include transition to and from government is another pathway we should curate.


Thursday, May 16, 2024

Not not the end of the world

Hello,


It's May 2050 and my name is Captain Jon Xander of the 17th Armed Division of 

the Ministry of the Future[1]. I'm here to give you a brief update of the past 25 

years' efforts. Summary take home, we failed to prevent the worst. Summary cause for hope, humanity behaves better than expected in a pinch[2].


Back in '25 some of the kids in XR realised that we needed more political and economic leverage against the huge momentum behind forces destroying the planet. Tipping points had been past, and it was clear we were already going to exceed 3 degrees global heating. At this point, within a decade, the antarctic ice shelf would be gone, and with it, all the coast cities of the US, and with them, the global economy.


Mass migration inland was also hampered by wet bulb temperatures across central US and southern Europe, latin America and sub-Saharan Africa, India and central China, sustaining over 50C for weeks at a time. Life was not sustainable here.


The ministry was created to coordinate economic incentives, to discount the future properly. Like open cast mining, the cost of restoring the environment to its original condition had to be paid forward one way or another - e.g. through carbon capture, reforestation there or elsewhere, from the get go.

But this wasn't enough. Hypocritical leaders were still flying around in corporate lear jets, so the undercover branch of the ministry was setup to bring those planes down, to set an example.


Models told us this was all going to happen, but those models had to steal cycles from the Hypercloud, the systems build by the Big Five Tech Companies to more precisely target marketing and advertising to you, and to persuade you to buy goods and services you didn't need, using ever more convincing (holographic avatars that look like your boyfriend, your mum, you) AIs. In 2030, these AIs ran on data centres that each consumed more electricity than a metro area like New York. They were overtaking the now shrinking fossil fuel companies as the root cause of the problem.


We tried geo-engineering - reducing solar radiation, spraying ice back on to the poles, even sails in orbit, etc. But that needed coordinated action, and the lack of that is the very thing that got us in this mess. Non-coordinated interventions moved monsoons thousands of kilometres away from where the rain was needed for food, and months away from the right season for rain in any case. Some actors even weaponised the systems to attack neighbouring states.


BY 2040, they were all history. The global economy couldn't sustain anymore than micro generation of energy with scavenged materials, no reliable power anywhere for computation or communications. How did we cope? How could we plan? With no Internet, of course, we just used The Clacks. But what about Machine Learning?


Here's an example (taking out Otis King[3] cylindrical slide rule). This was used by Computors back in 1930 to work out actuarial tables (and, sadly, ordnance for military) and these don't need a battery - this one was used in UCL stats department and is 120 years old. Works fine. 


So we can coordinate the routes for the sailers and fishing crews, plan the kelp  farms, and still predict the rate of biodiversity restoration through re-wilding and careful choice of trees to give us materials for housing, boats and clothes.


In the end, people are the key - 100 people with calculators like this can work out a route from earth to the nearest exo-planet. Of course, we don't need to as we shan't be seeing any space agencies for some centuries to come, except, perhaps, the wide open spaces between remaining dry land. Where luckily, we managed to save a couple of the really big libraries[4].


What do you think people miss most, today, in 2050?

The biggest disappointment for kids today is that 

they'll never get to play the electric guitar.


With thanks/apologies to 

1. Kim Stanley Robinson

https://en.wikipedia.org/wiki/The_Ministry_for_the_Future


2. Rebecca Solnit

https://en.wikipedia.org/wiki/A_Paradise_Built_in_Hell


3. Cylindrical Slide Rules

https://collection.sciencemuseumgroup.org.uk/objects/co148287/otis-king-calculator-cylindrical-slide-rule-england-1920-1939-slide-rule


4 The BL is this height:

https://elevation.maplogs.com/poi/euston_london_nw1_2ds_uk.541488.html


5.  Using the Cylindrical Slide Rule

https://www.youtube.com/watch?v=5_GjWHIo4Dk&ab_channel=JaapScherphuis


6.  Sunken Cities, Dobraszcyzk

https://onlinelibrary.wiley.com/doi/epdf/10.1111/1468-2427.12510


7. Five times Faster, by Simon Sharpe

https://fivetimesfaster.org/


8. Not the End of the World, Hannah Ritchie

https://www.nottheendoftheworld.co.uk/


Tuesday, April 30, 2024

Ring Law

A recent discovery of an as yet unpublished memo by the publisher Unwin, of Unwin, Wittering and Zigo, reports of an hitherto unsuspected encounter between Isaac Asimov and J.R.R.Tolkien in an hotel, where by pure chance, they were both attending international conferences by invitation. This would have been around 1949.

Asimov was of course well known for his biochemistry work at the nearby Columbia University, and was presenting his paper on "the melting point of benzoic acid and the feint possibility of room temperature superconductivity", whilst Tolkien was at the International Symposium of Runic Scholars, where he was going to speculate on "the possibility of artificial languages that could have their own symbolic script and may be a contribution to world peace". 

Unwin reports that the two gentlemen collided in the elevator as they were coming down after breakfast to go to their respective events, both clutching, as it happens, draft early manuscripts of there soon to be famous major oeuvres, the preface to the Adventures of Susan Calvin, and There and Back Again, Again, Escape from Mirkwood. In the collision, both bundles of paper fell to the floor, and, unnoticed by their owners, were inextricably mixed. 

Now we understand how it could happen that there were three laws that bound the behaviour of Hobbits, and that were seven rings for the mining robots, nine for the smart killer drones, three for the EVs, and one kept back for the POTUS. In a later story, Tolkien would revisit the three laws of Hobbits, and reason that Gandalf, an honorary Hobbit, perhaps was bound by a fourth (or more correctly, a zeroth) law, which subsumed all the others.

Asimov could never reconcile his Russian past with the idea that one human could possess such great power, and changed the end of the tale so that one ring was cast into the Yellowstone Caldera, and melted down, so that the three, seven and nine would now operate under true autonomy.

Fan contributed literature has tried to reconstruct the way the books might have been written without this terrible mashup, but little progress has been made coherently, partly because no-one has managed to find a convincing explanation of what happened to Susan Calvin. It has long been recognized that genre writers, even the very best like Asimov and Tolkien, were never much cop at character, least of all female characters. At least one partially successful attempt (you may have seen it adapted to Netflix) casts Gandalf as a woman (with a nod to Wilkie Collins' Woman in White).  

To the end of their days, both authors insist that allegorical interpretations of their work are ill-founded, and they were just writing escapist nonsense. Religions have flourished on less.