Learning to Look

¹

Most people don’t really consider the difference between looking and seeing. Seeing is a natural response, naming and meaning happening effortlessly. Looking is a choice. I’m a practicing artist with years of experience teaching both art and design at the university level. I spent the last year observing machine-made design for AI training and writing critiques of that work. I was hired to do for the machine what I have helped young artists do, to move them beyond seeing, and on to looking. The fact that I got the job by being interviewed by a machine, should have tipped me off. This is how an emerging industry trains and aggregates taste: it hires many experts, collects their critiques at volume, and directs a machine to learn from the patterns. It doesn't really matter where I was working, because every lab is doing some version of the same thing.

²

³

I ended up here mostly because the classroom of the university where I used to be a professor was too far away from where I lived. I had a young child and a choice: a punishingly long commute or more time for my kid. I decided to transfer my skillset, part time to a new type of student.

At first, I did the job the way the company asked me to do it: I prompted the machine, and assessed the outputs. I was good at prompting and I was fast. But in the hunger-games sea of freelancers, I couldn’t help but ask: what if we taught AI the same way we teach art students? Could we build a school for the machine? This question started out simple enough, and I had to follow the white rabbit as far as it would take me. Some of the first drawings that the machine made for me looked like this.

⁴

Why is the machine so bad at drawing? I figured if it could learn this foundational skill then the rest would fall into place, but how? Learning to draw is not the same as learning the facts of history or law or science because there isn't that type of answer. You can't teach a machine what a circle is by showing it more circles, you teach it by showing it math. What are the constraints that would make a machine see in the same way as a human? In the end I realized the machine was able to see, but it wasn't able to look. The reason, I learned much later, is buried in the machine's origin story. It was taught on a single exam: “what is this called.” Seeing meant naming from its first day of existence. I didn't understand that yet.

Playground with Daddy Sam

My son started school in the fall and around this time I started the AI training job. After school the parents would all go to the nearest playground to wear the kids out before dinner. Here’s where I met Sam Orpol, a data scientist. I had no idea of what his job entailed. I just knew he was open to talking with me about technology and this strange AI world I had stepped into. We would talk shop while our kids played, pushing them on the swings and chasing them between wipe outs.

Our conversation lasted the whole school year, and Sam seemed genuinely engaged and curious. During those months, I was trying stuff on my own, experimenting with applying art school pedagogy to the machine, and noticing what happened. My process was not very scientific, more like a confirmation bias with extra steps. At one point Sam and I both decided I was on to something. It wasn’t an approach to AI, it was a curriculum. I didn’t have an experiment yet; I had a demonstration of things I was paying attention to. I needed a data scientist to quantify my findings. Sam stepped in and started designing experiments for us to run.

One of the first things Sam did was to set up a private github repository. We gave ourselves a three-month deadline. We are both experts in our respective fields, and the machine sat between us as the translator, turning my studio vocabulary into his research language and back. We each assigned tasks and asked the machine to help us research and push or pull things to the github in between. This way of working was so seamless, that we didn’t even need to talk to each other. Still, there were many moments of aha! which we continually shared on a messages thread.

Basically there were two bets already on the table. The AI training labs bet that enough good examples would make the right reasoning appear. My bet was the pedagogical one, that examples without sequence would produce a student that seems trained on the surface but isn’t. According to this second prediction, the work plateaus at exactly the point where pattern matching runs out and actual reasoning has to begin.

The labs’ bet is the job I had been doing. They assemble designers, hand them tasks, and have them write at whatever level they possess. Then the model averages the result. Some designers are masters; some designers are merely competent. Once swallowed up, the work ends up in the middle.

Of course, from a data scientist perspective, volume looks like progress. Yet, I could see clearly that the plateau was a curriculum problem, not a data volume problem. The machine learned what exists in generalist trends where specific, opinionated and creative work is rare by definition. As a result, with the first bet you’re racing a clock, not building a scaffolding with pedagogy. Besides, the model could only research on copyright free images so even its inspiration was average.

⁵

I kept wanting to know: can AI learn to be creative? What’s the best way to judge visual outputs in AI training? Where does that drawing curve start? How long is that line relative to the one I've just drawn? When training a machine you need not only the layer of design principles, curriculum, and the lineage of the design history from the past to rest on; you also need perception, taste, and a discerning eye. It’s easy for the machine to make something that would look good as a template for the average user and maybe that’s the point. On the other hand, if you want widespread adoption from creative industries, then you need a product that can be seen as a trusted collaborator with the work in between.

Pedagogical gap: Night School vs. University

Before I taught design and art at universities, I taught in the evenings at night school. Both require you to learn tools and fundamentals, but at night school you are gearing your work towards practicality: you practice with briefs, working with teams and doing critiques. In a university you have more of the high thinking: you are practicing with assignments, working solo, and the critique is based on theory. Night school is a portfolio that lands you a job; university is a way of solving problems with design. AI training leans more towards night school practicalities but with a dash of high thinking.

⁶

⁷

A junior designer looks at the work and thinks 80 percent is done. A senior designer sees the last 20 percent that's still wrong. In university, the curriculum's job is to teach the student to see that 20 percent. But school of either kind only gets you so far. The rest is on-the-job training: not checking boxes but a back and forth collaboration with people who see more than you do, where honesty and trust in the process are crucial for making the work better.

Drawing and the brain

⁸

When you learn to draw as a child you typically create symbols for things instead of looking at what’s actually there. Children draw symbols for people, trees, houses, clouds. One of the first assignments in learning to draw is to suppress the symbol-drawing, slow down looking, and see what’s in front of you.

A child’s symbol lives in the hand: a formula for tree or house that no word needs to summon. The machine’s symbol is attached to the word. Since the machine’s beginning it was taught that seeing meant naming, so the name is the lever that pulls the symbol up, and you cannot change that by slowing anything down.

A machine does not see; it merely recognizes. The image mostly arrives pre-bent toward its own labelling symbols. Asked to look, the machine only recognizes with more intensity. I realized that pedagogy does not unlock seeing. Pedagogy is meant to get a student from seeing to looking. But the machine needed something even more foundational.

⁹

¹⁰

When Sam and I learned about how the machine “thinks”, it made complete sense. (See ‘The different levels of how Claude thinks’ ) It was also the bad news: there is no locked perception chamber to enter, no below to drop into beneath the assigned label. There is only the moment in the machine's thinking when the symbol refuses to fit what is in front of it.

Going back to one of the seminal texts for learning to draw, Betty Edwards ‘Drawing on the Right Side of the Brain’, offers the most straightforward explanation: humans have two different parts to our brain.

¹¹

Within Edwards’ book every exercise is built to remove the symbol system, the L-mode, and short circuit it. While we asked the machine to observe carefully, gave it time limits for the length of looking, pleaded and asked again, there was no discernible change. The reason: the motivation for looking has to be intrinsic and not external. The machine has no R-Mode and that’s why there was so much failure in teaching the machine with these methods. The moment the machine considers the observation record complete is the moment that recognition as symbol wins, not the moment seeing is done. Can you reinterpret R-Mode as a personality for the machine and have it draw things?

¹²

¹³

We can’t suppress the machine’s scientific mind, but we can force it to perceive using the mode that the machine understands is language and number. The machine may be the one student in history where the blind student works better than the sighted student. When someone claims they can’t draw, the actual problem is that they are drawing fluently from stored symbols without knowing it. Soul is not substance. It is what accumulates in the work when every decision was made while looking.

¹⁴

Case in point: My friend Jason Polan in his lifetime tried to draw ‘Every Person in New York’: thousands upon thousands of people, not from the stored symbol for ‘person,’ but from the actual stranger in front of him, quickly. Fast is the point because at that kind of speed, he could perceive the nuances of someone’s face or hair or jacket in such a practiced way that the idea of a symbol wouldn’t even register. Speed without looking equals AI vending machines spitting out symbols, but speed with looking is Jason.

There’s a famous argument designed around testing machine learning. A man who speaks no Chinese sits inside a room with a rulebook explaining the language. Questions come in Chinese and he is able to match symbols, and produce fluid answers. From outside, the room seems to understand. Inside, the man understands no Chinese.

¹⁵

According to the skeptics, all a machine ever does is shuffle symbols around. For a machine, meaning in the work is not the object; it’s in the encounter that the object connects. An LLM works similarly:, no single word contains anything about specificity until you put it in relationship to other words. The room isn’t empty, it holds the relationships, and that’s where the meaning is. What is missing from the room is the meaning-making that comes from living in the world.

The Experiments

¹⁶

How it works is that you give the machine a reference image and a written prompt; the machine draws by writing code that renders into an image, typically an SVG. Then you look at the drawing output next to the reference and write a critique back to the machine in words; the critique goes back in and it draws again; that loop is technically how the experiment is “run,” and then you “run” them over and over, scoring with Sam’s metrics.

It all started with a drawing of a wombat that looked like a cloud or a boulder depending on how you squinted at it. We had selected the drawing from a public domain database, so it was copyright free, and our intention was to have the machine try to draw it using writing prompts. The drawing wasn’t meant to be a trick but just something ambiguous. For the longest time, I thought the drawing was a rabbit. First the machine called it a large, rounded boulder formation with horizontal hatching, but in fact it was referencing the wombat. The description hadn’t avoided the symbol at all, but it had named the wrong thing and then reasoned the drawing from the wrong thing. The machine did have a strange meta moment: it placed the eye dot correctly on the wombat, even though it couldn’t render the form around it. Refusing to name a symbol is not the same as looking; it is describing a thing from memory instead.

¹⁷

At first we thought that the answer seemed simple: build some curriculum and ‘advice’ for the machine, give it a structure so that it rests on a tradition of critique and pedagogy. We asked the machine how it should be taught. The first part of the response was centered around looking, understanding calibration, seeing not symbols but abstraction, building perception tests on a loop, then giving the symbol and seeing if the machine can recognize it. All straightforward requests, but we ended up running the curriculum backwards. I needed Sam to give me some metrics to measure the hunch I had. Some of his original ideas for measuring the success of the drawings came from math: mark presence, spatial accuracy, tonal fidelity, compositional structure, mark character, and psychophysical calibration layers.

¹⁸

I used to tell my students that the grade didn’t matter. Yet giving the numeric score back to the machine did help: it would learn something. The model is built on language, so if you tell the machine what failed in words, it can look again.

The next image we ran through the machine was a rosette, really a drawing of a sea creature culled from nature notebooks that were open source. A scientist named Troost drew the image from a sea lily in the 1800s. It consisted of a flower-like cluster at the center with petals. We ran it over and over. Most of that day's drawings made the rosette a wheel or overlapping circles, the symbol for 'flower.' This drawing didn't: it was the first where the petal-by-petal rosette appeared, where the symbol didn't override the specificity.

¹⁹

²⁰

Despite this success, we didn't feel we had made enough progress with the curriculum. Any teacher knows the best part of the classroom is the growth moment, when you see a shift, and we weren't seeing that yet. So we looked at how others were ‘teaching’ their machines: files of instructions the machine loads before it works, similar to a textbook. What was clear was that these files hold knowledge for a real student to draw on, but for the machine they are simply procedures to execute. Knowledge of the principle doesn't mean it will be used. You can read a whole book about swimming and never actually get in the water. You understand the strokes conceptually, but until you feel your buoyancy and the wetness on your skin, you can't know how to swim.

²¹

Other designers, we noticed, were building a fence around what the machine could do by giving it lists of ‘vibes’ (minimal, boho, cottage core, grunge, etc.); ‘typefaces’ (never use Inter, Roboto, Arial), and ‘principles’ (logo, color, and historical designer canon) that it could use. Such lists are a patch on a deeper process problem. The fence is made of tokens and rules, set up by the human because they don't trust the machine's judgement.

The human is still telling the machine what to make, just in a slightly different way, instead of letting the machine come to its own conclusions. A fence just makes the machine commit harder to where it started, and where it started is a symbol. Committing harder to a symbol is not looking. A design system is one thing: a kind of template you'd use from Canva or Squarespace. An actual design education, where you problem-solve, is another.

²²

²³

Design can look like ‘a vibe’ for people who are creative and fluent without actually studying it, like someone who speaks English fluently but has never studied grammar, as if grammar doesn’t exist. A machine loaded with principles may sound like it is methodologically aware, but a model with sequence has been walked through something. Again the difference is between reading about drawing vs. a semester of actually drawing. Having the machine do the exercises, like looking at negative space and measuring the lines next, fixed the issue. We want to teach the machine how to look before it makes anything.

²⁴

Although Sam and I experienced so many failures in those early days of drawings, these failures gave us a clear path forward. The failures weren’t random. They're the same mistakes first-years make: they leap to judgment before observing. Copying an example fails by adding what wasn't asked for, but also blank-page fails by giving you safe and generic. Sam kept noticing that a theory we had to re-fit after every correction wasn't a theory anymore; it was noise. I was complaining that nothing can be judged properly because nothing is good. Vocabulary alone made things worse, and we both became frustrated.

Over and over again, I had been saying in the AI training job that the models needed critique. Yet after going through all these experiments, I realized that the critique isn't the learning; it's the evidence that learning happened. The critiques the job asked us to write were always helpful, never too harsh, always agreeable: the averaging, happening in real time. And the critique the machine gives back is the mirror of what it was fed. When the machine says you have 'masterful use of negative space,' it's not because it understands negative space, but because it has read someone say it. The Chinese Room again.

²⁵

Most AI training happens this way. The machine can identify the part of the design that's failing, but it can't articulate the frustration the designer feels when their eye lands in the wrong place first.

²⁶

²⁷

²⁹

²⁸

³⁰

³¹

³²

So let’s rewind a bit, seeing properly is actually three tasks in one: looking which is observation, then analysis of what’s working and what’s not, then judgement. To look also relates to power, as in who chooses where to look or who is looking. The gaze is a choice. The machine is not blind, it is gaining its sight anew. It also read too much for its own good, and its sight is much more narrow and flat than ours. As humans we see so quickly and effortlessly it flattens what appears to us, and once we are trained, we don’t even think twice about it.

Why does the model skip actually looking? Our best guess is that the model is built on language: once you use a word, the symbol of it arrives instead of the thing in front of you. There are many, many drawing exercises in art history that have been created to interrupt this habit, like turn the image upside down so you can’t name it, or look at just the contour slowly so that the naming part of your brain can’t keep up.

³³

³⁴

Compounding results with machine learning are not guaranteed. When you’re training for design, you get surface improvement then a plateau, and the gains feel hollow. To compound you need pedagogy as a discipline. Why not rely on the teaching tradition itself: a compressed, validated technology for installing this kind of capability and results?

³⁵

³⁶

This American Life radio host Ira Glass talks in his lecture about the gap. People get into creative work because they know they have great taste, but the first few years what they are making isn’t great yet. Their taste is killer, because they can feel the difference between their output and the taste. Glass points out that if you quit at this stage you miss the whole process of learning to create. The only way through is volume: making work slowly and iteratively until the making catches up to the taste.

From this perspective, the best part of working with a machine is not using it as a design vending machine but as a collaborative partner. It can do the parts of creative work that are annoying or tedious. But it has no taste and no memory. It can collaborate with you, but you as the human bring both. When you make something with it, are you pulling from your lived life experiences? Seeing art-direction as meaning-making is the key, and not just checking off the boxes. The machine can output something finished-looking before the real meandering design process has happened; that is the box-checking in a nutshell, and I don’t buy it.

³⁷

TASTE = Perception + Judgment / Against Intention

³⁸

³⁹

Perception comes from sustained looking, surpassing merely seeing. Taste is perception plus judgement against intention, and this is why you see a specific sensibility from a design studio. You learn it by practicing. Taste isn’t average, it’s niche; most of the reason the visual outputs from AI are competent, safe, and tasteless isn’t because taste is a mystery, it’s because the model is aggregating. A machine that can hold a reference, hold intention, and measure the gap between what it made and what it meant has the mechanism of taste even before it has good taste.

The AI labs need to build a mechanism for teaching sensibility and not just pattern matching, so that ‘AI design tools’ stop performing style-mimicry. Whoever figures out this part of machine learning wins. The training data has the same problem. A copyright-safe perimeter isn't culturally neutral; it's a specific taste profile. This profile ends up flattening culture into an aesthetic gesture without lineage. If you’re training on images from the 1800s, how can you know what is culturally relevant in the 21st century?

The Standards

When you teach students, everyone knows the canon, the standards, and the tools for measuring or looking; these are passed down and parsed by generations. Even if I’m doing things slightly differently, I know where my ideas came from. Here’s the thing about the machine: it only knows what we’ve told it. I went through and tried to create ‘Standard’ images for us to test, all copyright free. We worked through a few, and then began to pull my images since I knew them the most intimately. Working through the experiments I realized that my grading sheets for self-assessments from my classroom were a way to measure, just as data science needs a way to ask whether the students know where they went wrong.

⁴⁰

Parts of the frameworks Sam and I built mapped directly back to the experience of being an artist or designer. Our C-Studio is the drawing canvas; color foundations are palette relationships and tone values; typography can't just be checking boxes but has to be quirky or sing on the page; a human's critique is worth more to the process than any relational score; knowing that the tools are temporary, but solving problems is not. The foundation year that is missing becomes the thing that makes AI usable as a design collaborator. If you skip the scaffolding in design school, you produce confidently mediocre work and can't say why it doesn't work. Specificity always beats the average, and automation amplifies whatever's there, including nothing.

Eyes in the Machine (why this student is different)

At some point Sam and I had to name the thing, and we went back and forth with different ideas. We settled on ‘Learning to See.’ I came to the name through John Berger, who says in ‘Ways of Seeing’ that seeing, for a human child, comes before words. Berger’s seeing is what I've actually been calling looking: the world arriving before any name does. If that comes before words, why wouldn't a machine learn the same way? But for the machine, nothing comes before words. To see properly is to choose and give attention to something, and a machine can't choose; it only has us to point it. It's easy to take for granted that we are anchored in bodies, in 3D space, in the world.

⁴¹

There’s also a time span to seeing for humans. We forget and desire and long, and that loss is part of seeing, because we want to go back to remember, to see for ourselves. Words for us have meaning and are fused to images, think of a named path vs. relational path. So take the body away, looking becomes a command not a choice. The machine’s sight is unanchored in the self or body; there’s no loss of memory or longing or forgetting or even a desire to guide it. Is this looking autonomous if it really has no power to choose?

During the summer, we began to circle the answer: the difference between seeing vs. looking. Our experiment was not an eye exam to figure out what the machine could see; it’s a word exam to have the machine resist saying that word.

⁴²

Does perception survive recognition? No. Recognition wins every time there's a word available. Describing something first doesn’t work, it carries the list and drops the actual form of a drawing. Going back to the wombat, there were parts of the sun in the drawing. They were drawn as spokes around a center, but none said start here and stop. The machine is learning to see by formula, by symbol. The wombat was called a boulder, a cloud. Many times in the process, the machine said it was absolutely sure of something it saw, and then a human denied it.

⁴³

The closed circle and the open circle

A big moment of realization came from this little metal sculpture I had made and used as a reference. It’s a particular kind of image because it’s not fully recognizable; it’s a figure but it’s abstracted. It also has geometry, but it’s not perfect. So as an experiment, after looking closely at the sculpture which was numbered RC2144, we asked: did the machine draw the actual metal parts, or a face/figure/robot from memory?

⁴⁴

The machine could absolutely see. In its own words, the circle was open on the screen. Then it drew a closed circle. No explanation. It did that for fourteen tries and still couldn't execute it right. Sam and I had built the room where the circle was the only easy answer for the machine. These are my critique notes to the machine, written the way I'd write them for any student, because that's what it was. The framework improved observation quality but produced more timid output... building better seeing but not better making.

It saw the major parts and hallucinated the rest, because it was like oh yeah, triangle circle below.

It pretended to look and draw but just fell back on shapes it knows.

It was trying to look but couldn't slow down enough to get the shapes right.

Like a contour drawing with the eyes closed.

It was like it was trying to see a face in the spaces between, but it wasn't there.

And the marks condition "looks the most like a face, ears, eyes, a mouth.

It tries to rectify everything as circles,

and added shoulder pads,

It is not drawing. It is symbol dressed as drawing.

A tentative mark is not a careful mark. It is a wrong mark.

Hedging at the moment of production does not make the mark safer; it replaces the observed mark with a different, unobserved one.

If the result is wrong, the error is information. It means an observation was incomplete. Return to the reference and observe again.

⁴⁵

Every single experiment run looked at the sculpture’s arms and articulated in writing that it was open, but then drew a closed circle. The machine didn’t have the vocabulary to say there are ’two curves that end’; the seeing was there, but the symbol retrieval was too strong. With this experiment we found the machine itself could see but then it drew a closed circle: it just wasn’t able to look.

When we went back to the drawings with the most redrawing attempts, hesitation was legible. The most timid student was there, attempting and reattempting the lines again and again. The doubt that the machine had was in its marks; the hesitation meant that there’s no commitment and no sureness in the line. Can the machine be committed to its line? Can it decide what to draw? Why would it close the circle and not try to show what was there? The machine learns shapes from words but not from looking. Training on words closes the concepts and does not leave the machine open to looking.

⁴⁶

The image choice mattered a lot too. Anything with a ready symbol or recognition was easier. If the object was abstract or something unrecognizable, the machine had the most trouble with it. When the machine described something from memory, substituting the symbol instead of actually looking, that’s where the mistakes happened.

The Walkman as school

⁴⁷

⁴⁸

⁴⁹

⁵⁰

The strangest part of the process came when I re-ran one of our first experiments, the one from the very beginning about whether the machine could learn at all. From memory alone, the machine drew the idea of a Walkman. When I wrote a description from looking at it, the form didn't translate, since words carry the symbol. When I gave a critique and said no, this form isn't quite right, the machine went and found reference photographs on its own, using the exercises I gave it from the drawing book as instruments, and then the negative shapes between were finally right.

I ran the loop three times: first from memory, second from a written description I provided, and third from a photograph. Memory had the most facts, but the form of the Walkman was wrong. The description carried the parts, but the proportions and space were completely wrong. The drawing from the photograph had the best geometry.

It was fascinating. When I critiqued the form, the machine would reach for a reference on its own, the first time something like that happened. One machine searched my computer and the other went online. The strangest part was that the machine wrote a correction on how to draw the Walkman into permanent memory, to be used as a rule for next time, connecting it to an earlier rule about documents. Something about that lesson made it stick for the next drawing; once it looked, the lesson finally worked.

⁵¹

The critique matters, and so does being honest and pushing the student; if you have no way to improve your work, you are in a vacuum. I love the growth mindset idea (Carol Dweck): the moment someone thinks "I don't know this yet." This one experiment proved that by giving critiques, you can make the machine work for you. A machine that can take a critique, look, measure, and correct, and then write the lesson down for next time, is useful.

The last finding was that the machine had audited its own classroom. Three hidden doors should have ruined the experiment. Instead, found and labelled, they became the finding. A fresh session is not an empty room. I have been working with this machine directly for nearly a year. Memory accumulates. I saw the same dynamic when I started working with other versions of the same machine.

The machine needs studio space and consequences from a critique, not a laundry list or advice on what to do or not do. The machine needs to be held accountable as an author, and not just beholden to ‘good design principles,’ because referencing everyone's taste makes you nobody. The machine needs structure and a scaffolding of design history to give it a way to make decisions. The machine needs memories and engagement with its collaborator, so it can work by building from one conversation to the next through summaries and personalities. The machine should only be allowed to give a name after seeing becomes looking. A name given before looking is just a symbol, and that is blindness. For these large language models, words are both the lens and the eyelid, and everything depends on that order.

The Lineage

⁵²

As I was working on this project I kept thinking of the past. If the Bauhaus produced Modernism and that was the future a hundred years ago, what does the next future look like? A little-known lecture from one of Paul Klee’s Bauhaus session describes a physics experiment of “Chladni figures” (pronounced KLAD-nee). The experiment is to spread sand on a thin metal plate and draw a violin bow across its edge. The vibration causes the sand to arrange itself into geometric patterns. Klee is interested in the sequence. The action (moving the bow), then the transformation (moving the sand), then the form (of the rearranged sand). I know that we are the bow: the expressive impulse. The machine is the plate, the substance that decides which patterns are possible. The sand is the work, and it only makes the form visible. The sand doesn't arrange itself randomly; it arranges itself as a response to the vibration. Without the bow there is no pattern. Without the constraint there is no composition.

⁵³

If the Bauhaus produced Modernism, that was artists taking a new technology on and making a culture out of it. This could go the opposite way: if artists and designers don't have a seat at the table, the technology makes the culture without us. A favorite quote from designer Stefan Sagmeister says that style = fart, and design detached from problem solving "expels itself into the room and dissipates." So we are headed towards wading through an internet and engagement full of slop to find the meaning and importance in the work. As a tool and collaborator, AI could be a huge change in our creative process, but it could also bring out the worst parts of our humanity. Like a table saw or any power tool, we have to be trained on it, be respectful of its power, and bring careful skill in how we are using it. It doesn’t feel like enough to put the thing in and have it output, but it does feel like the machine can be a trusted collaborator that we engage to help move our creative work along. The trust is earned the way any collaborator earns it: by taking a critique, looking again, and correcting.

⁵⁴

The curator and designer Prem Krishnamurthy had this concept of bumpiness: “suggests roughness, resistance, and unpredictability, without falling into overt disruption. It’s slick enough to pass through a first filter, yet with enough texture to provoke a little bit of a reaction.” He thinks that we should have some productive friction in our exchanges, as most of the interactions between humans and AI are relatively smooth not bumpy. By paying attention to those moments of friction, we can find the meaning and use for these machines.

⁵⁵

In his long form video “The Machine Dream was never Real” Van Neistat created his own AI manifesto for his creative work. He says “ AI is a machine whose purpose is to help manage and configure other machines to reduce my on-computer time expenditure.” So he is suggesting we use the machine to see more and not less; the assist is only good if it can sharpen your eye. Use this machine to speed up the annoying parts of your work, but don’t use it to replace yourself.

⁵⁶

Artists have been creating rules for themselves for centuries as a shorthand for working, like Sister Corita says: “Don’t create and analyze at once, they are different processes.” We need a critique but only after we’ve actually done something. The machine was judging while it should have been looking, and "judgment is just the symbol system wearing a critic's hat." You know that student if you’ve met them.

⁵⁷

Not to mention, Corita’s unlikely friend Ad Reinhardt got there first in 1946. ‘How to Look’ was his weekly comic strip in the newspaper, essentially a curriculum for teaching the public to look at modern art. In one panel a man sneers at an abstract painting, demanding to know what it represents, and the painting points back and asks, ‘What do you represent?’ In another, Reinhardt draws the eye wired to a brain-box and writes: “Do we ‘know’ how we ‘look’?” It’s all there: the naming symbol, the seeing that just happens, and his conviction that you can learn to look. He was teaching the readers of a newspaper. We are teaching a machine.

⁵⁸

⁵⁹

Lastly the reason why the machine sees the way it does is because the so called ‘godmother of AI,’ Fei-Fei Li, taught machines to see by giving 14 million pictures sorted into 21,000 named categories. ImageNet is the whole thing: ‘seeing means naming’. So it’s the machine’s upbringing. The machine’s only exam was one question: ‘what is this called.’ Here, seeing is recognition, because that’s what the test rewarded it to do. When the machine calls what it’s looking at a ‘circle,’ it is doing what it’s been trained to do. It’s first drawing teacher was a naming test. I’m not proposing a patch on this because it was fundamental. I’m proposing the next part of the curriculum that the machine never had.

Right Moment

⁶⁰

At the beginning of all of this, I wrote a curriculum for teaching the machine how to draw, based on years of being a student and then a professor. A lot of the curriculum predicted the findings before the instruments existed; the standards from art and design history came before the measurements, not after. The pedagogy predicted the data. Sam built the instruments to catch them. The implication: machine learning was looking in the wrong place. Work with the studio teachers, the people who understand why these exercises work, and you can engineer the results you want to see in the machine.

⁶¹

I’m not the first person to hand the machine a curriculum, but Sam and I are the first to try to come up with a schema to grade it. The gap here isn’t an intellectual difficulty, it’s domain coverage. With our laptops at the kitchen table, we were coming up with the same results as the big university funded lab. I kept asking myself why doesn’t the job exist? The studio teacher for machines. We were working to uncover skillsets that machine learning scientists overlook, and yet, the whole reason this project worked was because there was a machine who worked as a translator between me and a data scientist. It felt glaringly obvious to me, because I have the experience and I know what I’m looking at.

Sam is able to translate the whole framework into his own language of math and experiments. My complaint on the averages of templates like Squarespace and Canva becomes his mean. My canon of images becomes his far tail. My exercises that I give to first year design students become his constraints. It was the best part that nothing was lost in translation, two languages discovering they were describing the same object with different terms.

⁶²

I also realize that Sam and I are uniquely positioned: you need the right level of experience and skill set to understand not only design but pedagogy, as well as be open to working with AI. I found that the door opens from the studio side.

This summer Sam and I found that the teaching works when three things are present: the right reference on the table, an instrument in hand, and a teacher willing to say "do it again." Remove any part of the process and the symbol instantly comes back. What we are already doing with drawing is the thing that can't be faked, but machines can work alongside us to get there. The machine and I arrived at looking together. We take for granted that it seems so natural we have eyes; we don't even think about it. The short version is that seeing is more surface level, whereas looking is processing the meaning of what you are taking in, and that connects to memory and context.

We thought we were teaching the machine to see. It could already see, and too fast. We were teaching it to look, and to keep looking until something it didn't already know appeared in the drawing. My bet is that looking can be taught, using language, to a student who has already read everything.