Thursday, December 06, 2018

above your station

"Bear up, Paddington", said Mrs Brown. "Why the down face?".
"It's family again, Mrs Brown. Aunt Lucy wrote that my cousins want to come and stay".
"what could be so wrong about that, Paddington? I mean it's not as if they're wild animals, is it?"
"No, Mrs Brown, but they are all rather odd".
"Tell us about them, Paddington", asked Mrs Bird, who had just come in from changing the oil in the land rover.
"well, I'm not wild about them - there's cousin Victoria - she isn't too bad, so long as she's not enamoured of another female bear. Then there's cousin Teddy, who used to be called Mary-le-bone, but underwent endangered species re-assignment surgery, as she was so bullied about her name, and was thoroughly stuffed. Last of all, there's cousin Euston, who claims he's neither one thing nor another, but is a bear-faced liar - a case for a good hard stare, if ever I met one".
"oh dear" said Mr Brown. "But perhaps they will all be fine after a good marmalade sandwich or two?".
"No" retorted Paddington. "That's just it. Victoria hates marmalade - won't eat anything but sponge. And Mary, I mean Teddy, has given up eating altogether. And Euston just gets in on jam after another. I live in dispair".
"but can we bring them to school" cried Judy and Jonathan. "It's not every family that has a sleuth of lgbt bears at home!".
"paddington's not gay" cried Mrs Brown, but Paddington was already blushing to the edge of his hat.
"political correctness gone made" muttered Mr curry, who had his ear to a cup against the party wall and had overheard everything.

to be discontinued...

Thursday, November 08, 2018

"but would you let your daughter marry one" - deconstructed

heard this phrase from some article about israel/palestine - you can probably guess but it could be from a guy (yes a guy) from any extreme group there - let's deconstruct this shall we?

"but would you let your daughter marry one"

but - so the phrase usually follows some apology (I'm sure they are all fine, but- or even I'm not a racist but). i.e. the speaker is a racist.

would you - why does the speaker think their view is like the listener ("you")?
how about my answer is "none of your business" - but also, why "would"? it isn't up to me.

let - see "would" - someone who wants to marry is by definition legally (and possibly religously) allowed by the state (church, mosque, synagogue etc), so it isn't up to the speaker or listener if the daughter is allowed - it would, indeed, in most countries nowadays be illegal and in many religions unethical, immoral and untenable to "not let" someone follow their heart...

your daughter - raises to questions immediately - 1. do you know (assuming it is a guy speakning or listening) that it is your daughter - there's no guarantee without dna testing. 2. why single out female children (actually grown up since they are of marrying age)? what's worse about a daughter than a son marrying "out"? is there some sexual deviance about this, or is there some incestuous unhealthy obsession or is there some property rights question? all mediaval bullshit.

marry - marry, why now? but why is marriage a "step" too far? and what if its same-sex marriage, does that make it ok, better, or worse? if the marriage is to someone of a different (or no) faith, then perhaps there's conversion going on (either side, either direction) or abandoning of religion, or, hey, guess what, the other "side" might be more tolerant of mixed religion marriages - given the above (daughter) and matriliear rules on the religions in the source region in question, what exactly is the "concern"? precision please, in your bigotry.

one - reduce the daughter's chosen loved one to "one", as in "any one of" - like people are interchangeable if they aren't the right kind of people? great, reveals what is really going on which started with "but" and ended up only 7 words later revealing inherent lack of regard for some people as humans.

I'm sure there are more things one could extract from this phrase, but it sure is revealing of the speaker's mindset.

Monday, October 22, 2018

letter to my MP (and any others listening)

Dear Keir Starmer,

I'm afraid I've had to leave the labour party - I am dismayed by the failure to show solidarity with the young, with working people, and with our international neighbours - the deliberately under-enthusiastic stance taken by the party about the misguided campaign to leave the EU is simply no longer acceptable. As a party allegedly representing those very people, it is time to take a stand for those people, instead of playing transparent, childish, but destructive Westminster games. Those are the very games that lead to mass disengagement from politics. Democracy is weakened by them.

As for the non-attendance at the march in London on Saturday, what was that about? As an MP in a constituency which is mainly pro-Europe, showing support for constituents should be above party politics. 700,000 people and counting bothered to turn out - it was a great event. The last two times major parties ignored movements like this led to Thatcher's downfall due to the poll tax, and Blair's disgrace due to the war in Iraq. Do you really want to make it 3?

Yours sincerely,

Jon Crowcroft 

Friday, June 08, 2018

What if time was a disease?

Maybe a virus you can catch - before the virus emerged, there was no time - everyone lived in the moment. Then there was this instantaneous pandemic, and since then, a lot of time has passed.

That moment? maybe that was leaving Eden.

Tuesday, March 27, 2018

duplicitous singularity

re-reading the Altered Carbon trilogy, I'm struck by a fundamental philosophical with re-sleeving (upload)  - while the book runs with the paradigm of digital recording of the essence of consciousness and skates over thin ice about embodied intelligence (the meat is just a machine the s/w of a human being runs on), there's a separate idea of continuity; as with perfect forward and backwards secrecy, how is the new running copy anything to do with previous copies, and how do previous copies anticipate the new copy? they don't  - they are copies. so philosophically, they aren't the same person, they are separate instances. but worse, if there's any notion of "eventual consistency" in how anticipation (and memory) overlap and interleave, then this just doesn't work at all. not one bit.

Thursday, March 08, 2018

What isn't AI?

so data science and its typical tools aren't really AI - machine learning, even deep learning, even generative adversarial nets style deep learning still isn't intelligent, though it sure is artificial - its useful, but claiming that a classifier trained on zillions of human-labelled images containing cats and no cats, is recognizing cats is just stupid - a human can see a handful of cats, including cartoons of pink panthers, and lions and tigers and panthers, and then can not only recognize many other types of cats, but even if they lose their sight, might have a pretty good go at telling whether they are holding their moggy or their doggy - how? well, because humans (probably) have a large collection of tools evolved (and trained) in the brain, and the brain is embodied, and so has perception, interaction, motion, sense of things light touch (how furry is that fuzzy looking cat's tail, how does a cat hold themselves when prowling, playing or just chilling... etc

these tools operate at many levels - some may just be context/recall, some may effectively be analogue programmes that model gravity or other physics things (stuff games software writers call "physics models"), and some may very well look like artificial neural nets (things to de-noise complex signals, and to turn moving images (the retina has gaps and doesn't refresh infinitely fast) into representations that let you find objects and name them (that cool cat is a songby squeeze :-) )

there are feedback loops between the "low level" perception stuff and the high level models so the models are (surprise surprise) nature & nurture...but prefacing learning with a model is going to help with unsupervised learning a lot - if we understand 3D, motion, gravity, materials (skin, bone, fur, muscle, fat, what we're made of, and what these other things are made of, wood, grass, mud, water, air etc) etc, then we don't have to see zillions of images of an X, because we generalize from one or two images to lots of views we'd expect an X to look like when its curled up asleep, or jumping up 3 times its height to catch a bird.

So there's a higher level still than al that, as if that wasn't complicated enough for you: humans (most animals) are pretty autonomous - they have goals (tropisms - find food, find partners, survive, enjoy, avoid pain etc), and they have some slightly less obvious tools like curiosity, imagination, creativity - all with a smattering of randomness. These can help seek out diverse input (and create different interactions) so our low level perception & Interaction are constantly refreshed, and our models are updated by challenges (think, scientific method & falsification & parsimony/occam's razor etc).

Then there can be another level still - things like self-awareness, consciousness,, beliefs, neuroses, and even "taste"/aesthetics, and of course, we are social beings, so ethics and theory of mind, and daft stuff like religion and manifest destiny and other collective psychoses. Ghost bugs in the machine.

Monday, January 29, 2018

data science, ml & ai - so, crates

A young robot's primer on the origins of her learning and intelligence.

Data Science mediates between evidence and knowledge. it is as old as humans.

Plato explains the theory of forms through the allegory of the cave, wherein prisoners can only see
the shadows of puppets on a wall, cast by the light from a fire behind them. the prisoners attempt
to infer what is really going on based on what they see.

The analogy beautifully captures the separation between the imperfect nature of
evidence, and the complexity of the model one might form based on said evidence.

Data (from the Latin for "given") is often flawed. our models (forms) are usually limited.
We have developed natural philosophy, and scientific method, to arrive at the best models we can -
e.g. by parsimony (Occam's Razor). however, for millenia, the job of sifting evidence and creating
models was combined. experts carried out (at least) two tasks.

Statistics emerged as a way to summarise and model the data itself, specialising the business of 
understanding the evidence itself, so that the form of models is abstracted away from the details
of the data (to some degree).

Now, we have moved to a world where the formation of models itself is, to some extent, part of this
new science. in varying degrees, the mechanisation of this process of model formation has moved on.

Machine learning packages up the business of spotting
which puppet is which, and even what the plot is.

Artificial Intelligence attempts to move this on to
figuring out it is a puppet show, inferring that there are puppet masters, 
and deciding what they intend the shadow play to mean.


In the realm of science fiction (The Matrix, Dark City, etc), there are actual puppet masters. In
the realm of superstition, these masters may even be adversarial. Post enlightenment, we usually
assume the puppet masters have no intent, but merely represent a more complex world, for which the
puppets and their shadows give us a chance to understand. Popper's model of objective knowledge
suggests that scientific method allows us to ever improve our (approximate understanding.

The mechanisation of this process starts with the problem of industrial scale data gathering,hence
with industrial scale organisation of society. Hence statistics for things like insurance (actuarial tables)
had to be computers (by people initially; and many other planning tools.

With the emergence of computers & the digital age, we gather more data, store it and can process it
more readily. Excitement over "thinking machines" led to the notion that human intelligence could
be understood through computational models. Early on, Artificial Intelligence researchers realised
that a number of sub-challenges exist, starting from understanding more complex evidence from the
senses - hence image processing and understanding objects and scenes was important from the get-go, for a
robot, for example, safely to navigate its environment. Social intelligence requires communication,
and humans use (largely) speech, so natural language processing was a requirement. Other, grander
challenges concerning consciousness and imagination were envisaged, but there was enough to be
getting on with. Indeed, too much at the time. It wasn't until recent decades that the tools and
techniques scaled up to tackle these problems with comparable efficacy to humans. 

Between the 19th & 20th century advances in statistics, and the discipline of Machine Learning,
there are a great many steps. We are not just concerned with quantities of data ("big data") but
with quality and structure of said givens. There's a huge range in the effort put in to make sure
there are consistent records kept in the simplest system. What those records represent is
also drawn from a wide range of complex structures (including no structure!). The advances in
mathematics and statistics are now coupled with advances in computing platforms to allow
exploration and comprehension of these properties of the data. As we go further down this road, we
start to infer what the data tells us about the world it comes from. This is not easy, as Plato's
allegory illustrates.This is also where we move from statistics to data science - we now have a
discipline which makes (falsifiable) predictions about the shadows and the puppets. Others can see
if out predictions remain accurate, or deviate from what they observe. We can use these predictions
to help look for explanations of what is going on, and choose between different explanations. That
process itself is to an increasing extent also being mechanised (automated). 

However, the biggest advances are really often still the old techniques running on affordable,
very fast hardware with a lot of storage. Rarely do these techniques surprise with something that 
resembles general intelligence. That said, the tools are clearly increasingly very very useful. 
Domains where we have potential to have a huge impact are added every day, and as we gain the
capability to apply data science on new combinations of evidence, the unpredicted or unexpected 
will no doubt emerge. While we started this note with a discussion of the emergence of a science
from the shadows, many of the key applications of the science concern data about us - humans. As we
make more observations, and automate processing and decision making concerning humans, we mustn't
lose sight of that humanity. After all, we are not the puppets.

As Conan Doyle said thru his memorable character Sherlock Holmes: 
"Once you eliminate the impossible, whatever remains, 
no matter how improbable, must be the truth."

---------part two--------

statistics aren't what they used to be....before and after Bayes...and before and after Hinton....

two advances in statistics combine with lots of fast computers and data to yield a lot of the machine learning and AI excitement - that's the way that we update our knowledge when we learn new evidence, due to the reverend Bayes, and the way we avoid having very much explicit domain knowledge in our machine for learning, due to artificial neural networks and "deep" learning 

first off, we need a bunch of data - and typically, we need some clever people to label that data. so now we need to talk about domain knowledge and expertise, and how machine learning and AI has so often been parasitically dependent on human intelligence. this is where we "crowdsource" (or already have lying around) a bunch of data that has aready been classified in some senses as "A" or "not A" - 

so lets work by example: lets say we want to learn a classifier for images with cats. and we have many many images (e.g. from google search or from some other available data) that has been tagged with whether or not there at cats in the image. so now we "fire up" our very simple neural net on this data, to train it in what is "cat-ness"  - input at each step is an image and the label (cat or not cat). the neural net is, effectively, a big table where we write things in on the left hand side columns, and copy values through to the next columns if they fit some value so far (are near enough), and not if they don't. When we get all the way out the far side, we have a column full of numbers and one box that says "is cat" or "isn't cat". we compare that to the input and if we're right, we increase the values in the boxes in the table, and if we're wrong, we subtract from those values. [n.b. this is a gross simplification of how neural nets work missing out hidden layers, convolution, and piles of important other tricks that make things work "better")...] Another way of thinking of the "layers" of the neural net might be like a series of combs, each with different length (adjustable) teeth, which let things flow through to the next comb, and when the answer pops out the end and is right, we leave combs' teeth lengths alone more, and wrong, we adjust their lengths more. sort of:-) The adjustment process uses a technique called stochastic gradient descent - one way to picture that part of the process is like snowboarding many times down a complicated slope, adjusting which way you go on each trip, til you get the best ride. or skateboarding. 

we do that for millions and millions of images. whether the cats are black or pink, furry or bald, the result is a table/box that is really quite good at spotting cats (a trained classifier) - we can then put that table in (say) a camera, and now the camera will be able to say "cat" or "not cat" and tag pictures itself without a human.

we could to the same with (say) machine translation. again, totally dependant on work already done by people, say we have a lot of text that has been translated (e.g. copies of Dickens' novels or travel guides or web pages, hand-translated between French and Latin and Greek and Basque and Korean. We can take the text and pull it into small pieces (words, phrases, sentences) and learn a classifier for these which "recognises" input phrase x in language A and outputs it in B...again, re-enforced over more and more examples, but once built, just a "table" to run things on your laptop...



What happens if we can't get labelled data? at least two reasons - we don't have time, or people just aren't good at this particular task - for example finding stuff in the asteroid belt, or explaining why a set of moves in Go were good (something people can explain in chess:). So one trick is to generate synthetic input to the neural net, where we "know" in the generator whether we put a pink panther in the picture or not. Another might be that there's something implicit in the output of the trained neural net that tells us if it is on the right track or gonig down the garden path - e.g. wins a game or not....so this is where Generative Adversarial Nets come in - where there are now two nets, one is like Socrates or Yoda, and the other is the "student"....

These techniques are very effective nowadays, although one interesting problem is that we're not entirely sure what the "features" are that they react to in all cases. Entirely different approaches to machine learning, involve taking a more explicit collection of rules (perhaps based in a model which we put in, in the first place) and just learning which are more or less important in which combinations - it isn't clear, but some research seems to be suggesting that there's ways to bring these approaches towards each other somehow...


There's much more to be said about other (and sometimes simpler, sometimes more complex) machine learning approaches, and about more foundational work on AI in the sense of trying to do something utterly different from all the above, which is actually understanding how humans (and other smart species) do so much without supervised learning. some the old and new theory for the latter concerns higher level goals & models & intentions....which maybe we just evolved to have, and sit at an entirely different level than the neurones and synapses in the grey matter and why it won't be just something emergent from more silicon....