A personal exploration of connections between creative computing, maker-centred learning, self-expression and social engagement; with a special twist on hacking fine arts.

February 11, 2019

Playful coding with a famous painting. Magritte

In this activity I try to show how we can have fun using coding to hack or play with a famous painting. Specifically, I used Scratch to do some playful coding with the painting "The pilgrim", by René Magritte. Magritte is an artist that children like very much (like all surrealists) because of his ability to unsettle you and make you wonder what you're seeing.

The programming in my project is very simple. It allows to move the head of "The pilgrim" with the arrow keys and make it coincide with the body (in the painting they are separate). You can also change the head and make others appear: the same face covered by an apple (a reference to another painting by Magritte), the Scratch cat, or Picasso's face.

In this project we are exploring two of the many things we can perform on a painting through creative computing: (1) to make an object in the painting move, and (2) to make appear new objects that were not in the painting.

Even if we use images of artworks for educational purposes, it is very important that we always cite the works and the authors.

To move an object in the painting, first we have to turn it into a "sprite". This means that we have to cut it out from the image, and fill the remaining space so that the cut is not noticeable (content-aware fill). In the image below you can see the original painting, the version without the face, and the face.

Although cutting images with the Scratch drawing editor is feasible, if we want to do a precise work, then it is not recommended. To do this kind of tasks we can use free and open source software like Gimp. There are also online editors like Pixlr. Gimp has a plugin called "Resynthesizer" that allows you to select something you want to remove from an image, and with the push of a key, it's gone without a trace.

When you have completed these steps, programming with Scratch is very easy. Simply put the painting without the face as a background (or as a sprite in the back layer) and then add a sprite that is the face you have saved in a separate file. From then on you can move the face, modify it, add sounds, or anything else you can think of.

I've added some buttons to the project that allow you to change the face. I do it through "costumes". The same sprite has several costumes with other images that I have added. Check the project and the scripts here, and feel free to remix it.


There are many ways to reimagine this activity. Take a look at other Magritte's paintings, you will see that they are very thought-provoking and will give you a lot of ideas. Let the children explore them. Also think about how to add interactivity with the physical world. For example, there are some objects that appear recurrently in Magritte's paintings (hats, apples, pipes, umbrellas). You can play with real objects and the Makey-Makey kit to make them appear or disappear from the paintings on the screen.

February 4, 2019

Imitating Jackson Pollock's abstract art with Scratch

This article is the first in a series that will explain how to play at imitating the abstract art of some painters by using Scratch.

We begin with Jackson Pollock's abstract expressionism. My Scratch project is inspired by his work "One: Number 31" (1950), on display at MOMA, New York City. 

Pollock, well known for his "drip paintings", is a very popular artist and proof of this is that on the Scratch website we can find dozens of projects inspired by his works. 

The activity that I describe below is conceived to generate a quite realistic result (although of course very far from a real Pollock painting). For the activity you only need a computer (or some mobile device) and an Internet connection. 

Although the version of the activity I am going to describe is more appropriate for secondary school students (since they need to have notions of angles and concepts such as randomness), we could also simplify it and implement it with primary school children.

Step 1. Learning about Jackson Pollock

Of course the best way to enjoy Pollock is to immerse yourself in his paintings, standing in front of them in a museum. As I usually say, if you are lucky enough to have a nearby museum with Pollock works on display, the first thing I would recommend is a visit with the children. They can take photos (if allowed) or make drawings, copying or inventing new works inspired by Pollock. But if the museum option is not possible, the Internet is full of resources to learn. For example, it is very interesting to know how he painted directly with the canvases on the floor (see video above), what came to be known as "drip paintings". The MOMA website is also a good place to learn and enjoy the 86 works that are shared online.

The painting I have chosen is huge and occupies an entire wall: 2,69 m x 5,30 m.

Step 2. Painting a first layer of lines on the virtual canvas

The way I propose the activity, the programming is divided into two main parts. The first consists of painting thick lines and large areas of color. These lines later will be almost covered when in the second part we paint the network of thin lines, but they will generate some spots and shadows that are very useful so that the final effect is realistic. 

Scratch is a free programming language and online community where you can create your own interactive stories, games, and animations. On this occasion we use it to make an interactive artistic project.

In Scratch 3 there is an extension called "Pen" that allows us to draw on the stage. It's as simple as using the Pen Up and Pen Down blocks and moving the sprite around the stage. This is how we can draw a thick curved line (with random thickness and random transparency): We repeat 50 times a small forward movement, and we turn a few degrees left or right.

Drawing a random, thick and curved line.

If we now repeat the previous script 50 times and add a random color (within a set of 3 possible colors), we will be able to fill the stage with thick random lines of different colors and transparencies.

 Drawing 50 thick lines with 3 colors.

Step 3. Painting a large network of thin lines above what we had created before

Now we repeat the previous process but drawing many more lines, thinner, longer, and with five colors instead of three.

Drawing a random, thin and curved line.

 Drawing 200 thin lines with 5 colors.

If we now put the two parts together, and we first draw the fifty thick lines and then the two hundred thin lines, we get a result that is totally random each time, but always keeps the same style. To give it a realistic touch and make it look like a photo taken in the museum, we can add a sprite that represents people looking at the painting.

Result of running the program 4 times.

When we program a project like this we are generating a lot of iterations. The first part of the program has two nested loops with a total of 250 iterations (50x50). And the second part has two other nested loops, with a total of 60,000 iterations (200x300). All this makes the execution of the program slow. The whole process of painting the canvas can take more than 30 minutes. Watching how it runs is beautiful (and almost hypnotic), and I've added jazz music in the background. But Scratch allows to run the projects in "turbo mode" and you can see the result in just a few seconds (Shift-click on green flag for turbo mode).

My Scratch project. Shift-click on green flag for turbo mode.

Step 4. Reimagine

Finally, some ideas on how to play with the project, improve it and remix it. For example you can start playing with the numbers (length and thickness of the lines, number of iterations, etc.) and see how the result changes. You can also play with the colors. I've chosen some colors very similar to those used by Pollock, but you can use any. You could also have several sprites painting at the same time, or add a sprite that "drips" small colored dots on different parts of the stage, to make it even more realistic.

January 29, 2019

Hacking Mondrian with Scratch and Makey Makey

This creative learning activity consists of replicating on cardboard a Mondrian painting, and then hacking it by using Scratch and Makey Makey. The final result is a physical device connected to a digital one. Touching with the hand the colors of our replica on cardboard, we change the colors of the original painting that we have on the screen.

For teachers, educators, or parents, here is a proposal of how to implement the activity (of course there are many variants and alternatives to explore):

Materials and resources 

A piece of cardboard, coloured wax pencils, a black marker, copper adhesive tape, a Makey Makey kit (board, alligator clips, jumper wires, USB cable), a laptop, and Scratch

Step 1. Learning about Piet Mondrian

If you are lucky enough to have a nearby museum with Mondrian works on display, the first thing I would recommend is a visit with the children. They can take photos (if allowed) or make drawings, copying or inventing new works inspired by Mondrian. If the museum option is not possible, the Internet is full of resources to learn. For example, the painting I chose is "Composition with Blue, Yellow, and Red" (1927), on display at Museum of Fine Arts, Boston. The museum's website has information about that painting, but also Google Arts an Culture let you explore the museum, or you can check websites like WikiArt, a visual art encyclopedia.

Step 2. Replicate or invent a Mondrian work on cardboard

The next task is to cut a piece of cardboard (or whatever material you think is most appropriate) and replicate (or invent from) the Mondrian work of your choice. You can use coloured wax pencils, markers, paints, etc. Once the artistic work is finished, our piece must be prepared so that it can interact with Makey Makey and Scratch. I used copper adhesive tape to mark some areas of the painting with which we will play. The areas covered by the tape will be like buttons that when touched will cause changes in the digital version that will be on the screen. A more effective and fun solution for children would be to use colored conductive play dough instead of copper tape. We could cover the whole area of some of the squares or rectangles with conductive play dough. If you use the tape, make sure it reaches the edge of the cardboard, which is where we'll attach the alligator clips. If you want you can also create buttons like the ones in my example ('random' and 'reset') that later you can also program with Scratch.

Step 3. Connections with Makey Makey

In this phase we have to prepare the connections with the Makey Makey board. Each of the areas where you have put copper tape or conductive play dough must be connected to the board, either by alligator clips or jumper wires. Makey Makey turns anything into a touchpad, so with the kit we can turn objects like bananas, coins, or anything conductive into a computer keyboard key to control onscreen gaming action, play software-based instruments, etc. In our case, we will turn our painting on the cardboard into a control with which to hack the real work of Mondrian that we will have on the computer screen. In the Makey Makey website you can find tutorials that you may find useful. But the Internet is full of videos, examples and detailed activities. Do a search if you need help or ideas.

In the example of the picture above I connected the blue area of the painting with the round "Space" pad on the Makey Makey. I also connected one end of an alligator clip to "Earth" on the bottom of the front side of the board. With this setting, every time I take with one hand the free end of the alligator clip connected to "Ground" and I touch with the other hand the copper tape in the blue area, I'm closing the circuit, so it is as if I were pressing the "Space" key on the keyboard. Of course now we need to program with Scratch so that each time this happens, the blue area of the painting on the screen changes color.

Step 4. Coding with Scratch

Scratch is a free programming language and online community where you can create your own interactive stories, games, and animations. On this occasion we use it to make an interactive artistic project.

There are many possible approaches to programming this project with Scratch. In mine, the idea is to put a background photo of the original Mondrian painting that we are hacking, or to draw a replica with the paint editor. Then we have to create a "sprite" for each colored area we want to play with. These sprites can be created with the paint editor, and must have the same shape and size as the area over which we will put them. Also, for each of them we have to create "costumes" with different colors.

The sprite for the blue area and its costumes.

The coding part is very simple. Basically we have to use blocks from the section "events" and the section "looks". The idea is to make the sprites change costumes when we press certain keys (or what is the same: change costume every time we touch a colored area of our cardboard work connected to Makey Makey).

 Scripts for programming the blue area sprite.

I programmed it without using the Scratch extension for Makey Makey. But Scratch 3 includes an extension specifically for it, making it even easier.

If you're new to Scratch, you can take a look at the tutorials it includes, or I also recommend this great guide: Creative Computing Guide.

Step 5. Reimagine

I haven't explained all the details for implementing the activity, but I think I've highlighted the key points. Here you can see the Scratch project I made. Try it, explore it, remix it.

With the same core idea you can imagine many other activities:

- Change Mondrian for other abstract artists.
- Instead of making the colors change, make the shapes move.
- Add music. Every time we touch an area we can change the color and play a sound.
- Choose a figurative painting, and when touching the represented figures on the cardboard make a dialogue appear on the screen, or make them move, or make appear figures from other paintings, etc.