Today I've found this beautiful photo by Lernert & Sander on Twitter.
It got me thinking about AI and image recognition. Wouldn't this image be a great test of how much your system actually "understands" about the objects it processes, and how much it just relies on some random features of previously seen images? Each piece in the photo is a small cube of some kind of unprocessed food. The cubes look fairly different from the food you typically see in real life, and yet humans shouldn't have a problem recognizing most of the foods: some kind of meat in the top left corner, avocado to the right of it, a mandarin or an orange next to it and so on. How would an image recognition system do?
I've decided to try and feed some of the cubes to the recently announced Wolfram Language Image Identification Project. Here are some of my favorite findings:
1. At least 8 different food types are recognized as emmental (a kind of cheese). Surprisingly, none of theme really looks like cheese - I haven't tried yellowish-white cubes which could be anything, just the ones which have some distinctive features: corn, kiwi, dragon fruit, banana, yellow bell pepper, onion, pineapple, and this weird thing that looks nothing like any fruit I know (any guesses?).
2. At least 4 different food types are recognized as soap: avocado, watermelon, melon (I think I see a pattern emerging here...), and meat (...nope).
3. Some of the results are completely wrong but still beautiful:
mushroom (likely an Agaricus bisporus cultivar specimen) labeled as "chocolate truffle"
romanesco broccoli as "nougat" (ok, I didn't know the exact name of this thing, but I was one query away from it - just google "fractal cabbage")
purple cabbage as "container" (and a pretty one it would be!)
4. Two cubes of pomegranate and mandarin got recognized perfectly:
5. Another several cubes got "recognized" conceptually: the system classified tomato, orange bell pepper, cabbage, and a citrus as some kind of food but had no better idea. You could put together a salad :-)
6. The picture of all the cubes together? Jelly bean.
7. And finally, the gem of the collection: a fruit which the system identified perfectly while a human (me) failed miserably. In my defense, I have to say that I've never dealt with a papaya and I don't expect to deal with one in the nearest future :-)
It got me thinking about AI and image recognition. Wouldn't this image be a great test of how much your system actually "understands" about the objects it processes, and how much it just relies on some random features of previously seen images? Each piece in the photo is a small cube of some kind of unprocessed food. The cubes look fairly different from the food you typically see in real life, and yet humans shouldn't have a problem recognizing most of the foods: some kind of meat in the top left corner, avocado to the right of it, a mandarin or an orange next to it and so on. How would an image recognition system do?
I've decided to try and feed some of the cubes to the recently announced Wolfram Language Image Identification Project. Here are some of my favorite findings:
1. At least 8 different food types are recognized as emmental (a kind of cheese). Surprisingly, none of theme really looks like cheese - I haven't tried yellowish-white cubes which could be anything, just the ones which have some distinctive features: corn, kiwi, dragon fruit, banana, yellow bell pepper, onion, pineapple, and this weird thing that looks nothing like any fruit I know (any guesses?).
2. At least 4 different food types are recognized as soap: avocado, watermelon, melon (I think I see a pattern emerging here...), and meat (...nope).
3. Some of the results are completely wrong but still beautiful:
romanesco broccoli as "nougat" (ok, I didn't know the exact name of this thing, but I was one query away from it - just google "fractal cabbage")
4. Two cubes of pomegranate and mandarin got recognized perfectly:
5. Another several cubes got "recognized" conceptually: the system classified tomato, orange bell pepper, cabbage, and a citrus as some kind of food but had no better idea. You could put together a salad :-)
6. The picture of all the cubes together? Jelly bean.
7. And finally, the gem of the collection: a fruit which the system identified perfectly while a human (me) failed miserably. In my defense, I have to say that I've never dealt with a papaya and I don't expect to deal with one in the nearest future :-)
