Indians are very touchy when it comes to, well, a lot of things but most vehemently so about our cultural icons. In the sacred mountain of worship of art and culture, Satyajit Ray and his magnum opus Pather Panchali are revered peaks. The outside world casting as much a glance in that direction usually results in rants about how nothing is sacred anymore, how sacrilege is the new hallmark of modern artistic practices, which are, needless to say, crass. It is high time we accepted that not everything is about Pather Panchali and Ray. Bengali cultural icons are not under attack. Not everything is about our great Indian heritage. (Keralites are nodding in agreement because everything is in fact about them and John Abraham.)
All hell broke loose when Aniket Bera, Assistant Research Professor of AI at the University of Maryland in the United States coloured Pather Panchali and presented it to the world. Bengali filmmakers wept. As a student of cinema and as what is known as a struggling filmmaker, I was enthralled. Not because I thought Pather Panchali looked better in colour but because it was obvious it was a harbinger of unknown possibilities.
The comments on YouTube spewed hatred on the act and raved about the sanctity of a piece of art, its timelessness et al. Bera had to eventually come up with a statement that clarified what was clear to begin with. It said he couldn’t understand the hate and that ‘This small experiment helped me to look into what possibly it would have looked when Ray made the movie with more pixel details.’
The footage had been automatically upscaled to 60 fps (frames per second), 4K and digitally colorized. Not that anyone was interested in what it meant.
What does it mean?
The original footage was 24 fps(frames per second). Making it into 60 fps is not a new idea, as we know. We regularly do it even on our phones when we want something to move in slow motion. There is no intelligence working to create new frames. Software would take the average of frames 1 and 2 and create frame 3. Bera explained in meticulous detail - “...The problem with older methods was that it was basically just averaging frames. There was no ‘new’ information, so even though the playback looked smooth, there were many visual artefacts like ‘ghosting.’ Ghosting is familiar to us. It’s when motion leaves a trail of the same image like a halo, following it everywhere. What AI does is similar to what a human brain can do. Also the reason why it is called neural network. AI can ‘imagine’. It doesn’t take the average, it creates new frames intelligently. That is, it learns aspects of the frames before and after and using this information, creates new frames in between”.
And what about all those 4K pixels?
Bera’s baseline was an online digital video that was 720p - that is, ½ K. The AI created 8 times the pixels to make it 4K. How does it do that? ‘The AI ‘dreams’ of pixels which never existed in the first place. The AI technology works like the human brain; it looks at millions of real videos (from different real-world datasets) and automatically learns to think as our brain understands the world around us. AI can imagine what it would look like, even with very low or poor quality videos, Bera explained.
Are you thinking what I am thinking?
Yes, this means that impoverished filmmakers who do not have the wherewithal for expensive equipment can indeed make 4K films. We are already using such methods without us perhaps realising it. The night mode in our phones uses the same technology, Bera explained.
Sadly, in most Indian film schools and film studies departments of most Indian universities, art is taught to have hierarchy. Real cinema is when a large number of people battle the universe to make a film. At the film school I studied Satyajit Ray Film and Television Institute (SRFTI), purists abounded among professors.
We had the great ‘das’. Tarko da (Tarkovsky), Bresson da et al. whom we looked up to. Da is a Bengali suffix for addressing, which means ‘elder brother’ and is used endearingly. Ray’s famous detective Felu-da might ring a bell. The department of cinematography couldn’t stop talking about how it was Subrata da (Subrata Mitra, acclaimed cinematographer of Ray’s films including the Apu Trilogy) himself who set it up. Amidst all this, a night mode shot or shooting a feature film on a phone weren’t options we were allowed to consider. Yes, we were all grateful for the Arri Alexas we got to shoot our student projects but stepping out of the institute, one would have to be an Ambani to afford such budgets.
I am reminded of the time Elippathaayam (The Rat Trap), a masterpiece by Adoor Gopalakrishnan, was being screened during my film student life. It was 35mm print and I was but aghast to see the level of deterioration it had undergone. Colours leaked across the frame, splotches sometimes blurred portions, shakes were migraine inducing in places. With the neural network AI, our film schools could be stacked with high quality copies of such films.
Speaking of Adoor, I was tempted to find out how a couple of his black and white films would look in colour. He was not impressed.
‘Why would I want to see what my black-and-white films would look like in colour? I made it in black and white, didn’t I?” True that I might not have the technical knowhow to make the AI do the work but interfaces are only getting easier. Despite his skepticism, in some years, in the comfort and secrecy of my laptop, I will be able to see his frames in colour, 4K and 60 FPS. In fact, some of the positive responses that Bera received were from parents who said that their children preferred coloured higher quality versions of films.
Prabhdeep Singh, cinematographer and a film school graduate agreed. “..We love to crib in our film history classes about how a lot of films were lost and how the state of archiving and condition of our prints are abysmal. But when something like this comes we don’t welcome the possibilities of what it could bring and start attacking it instead. I think this technology has the potential to restore some of those prints and many others in an economical way and not rely on expensive and bureaucratic systems of restorations that a chosen few films alone are able to receive.’ He said.
After all, it is, thanks to this technology that we were able to see the train, people, trees and leaves all clearly from one of the first films in history itself - Train Arriving at the Station at La Ciotat (popularly known as “Train arriving at the station”) by Lumiere brothers.
Gurvinder Singh, award winning director of films like Anhe Ghore Da Daan (Alms for the Blind Horse) and Chauthi Koot (The Fourth Direction), was not thrilled. “I don't see any need to change the frame rate to 60fps. That is giving this footage a 'video' feel. In an age when we all are looking for 'film' feel from digital cameras, this is totally undesirable. It has taken away the texture and experience of the original film. After all, we don't wish that the film was made in the video or digital age. It is a 1955 film and depicts that period. The materiality of the emulsion is intrinsic to the emotional feel of the film. Why would anyone want to destroy that? I don't think I will be able to see the complete film treated like this”, he said.
When Mughal-e-Azam was restored and coloured for re release in 2004, it had around 100 technicians working on it. The budget was never really revealed, perhaps for obvious reasons. Even before the AI coloured Pather Panchali hit the net, portions of Pyaasa and Apur Sansar had. These clips called ‘Imagine Colour’ were widely shared online and were the works of Sriram Raja, a filmmaker and FTII (Film and Television Institute of India) alumnus. It took him a week to render a one-minute clip, he told Times of India.
Compare it with the work of the AI. It coloured around 3 minutes in 7 hours. That is, the work that so far was done manually in one week can be done in 2 hours by AI - fully automatic. Why is this important? The first rule of filmmaking. Time = money.
Yet there is something disconcerting about the responses that these two colouring feats elicited. The manual colouring by Raja was seen as an experiment, a welcome point of view from a film enthusiast. A techie laying hands upon a masterpiece - not so much. Satya Rai Nagpaul, acclaimed cinematographer of the earlier mentioned Gurvinder Singh films said that it was ‘...a deeply violent act, that, like all such acts, has irretrievably violated the (publicly held & recognised) authorship of its creators, not on any discursive depth of its own, but on sheer technological entitlement.’ Award winning director of these films, Gurvinder Singh agreed that the upscaling was a ‘strict no’ for a connoisseur. He however thought that the colours were accurate and truthful and agreed with the parents who talked to Bera when he said that ‘it might make old b&w films more appealing to the younger generation who won't normally see old films for this reason. So as a marketing strategy to give a new lease of life to old classics, it might be good.’
One could argue that the manual colouring has an aesthetic edge - there are transitions from black and white to colour to constantly remind us that it is but an experiment. The colouring itself is not uniform, there is emphasis added to portions of the frame by a human being. Yet, we forget that these nitty-gritties can be taught to the AI in no time. This is exactly what scares artists. That they would no longer be needed when AI learns what they know, in a couple of years. We forget that it is impossible to do quality improvement through manual methods.
Not only will AI go ahead and colour all it wants, it will be able to make you feel what you feel you should feel.
Like one of the memes that surfaced after the AI coloured Pather Panchali and the purists’ wrath upon them emerged, AI might spit a Ray frame (colour, 60 fps and 4K) at us and say, ‘believe it or not, art is not sacrosanct.’
Don’t believe me, see these frames (the one used in the meme as AI’s response to the purists). A is the original image. B is the one AI upscaled. C is the one AI did on its own and D is the one it did after cinematographer Prabhdeep Singh gave it a palette of his choice to work with.
Take a look at the photos:
- 01/04
- 02/04
- 03/04
- 04/04
(Kunjila is a freelance writer and filmmaker from Kerala. She lives in Bombay. This is an opinion piece and the views expressed above are the author’s own. The Quint neither endorses nor is responsible for the same.)