Sabbatical over already?

Although I will not return to teaching until fall, I am afraid my first ever sabbatical is just about over. Apart from our car being totalled after my first week in Ohio, it has been a very positive experience. I was privileged to be part of the ACCAD community at The Ohio State University, where I spent six and a half weeks in person and collaborated remotely for the remainder of Spring.

The experience being played at the ACCAD Open House. Photo: Lexi Clark-Stilianos

An initial version of the full VR experience was presented at the ACCAD Open House on April 7. I have been collaborating with MFA student Raven Serenity Glover, they build most of the game mechanics. The experience is not really a game, but we still referred to the scripts that drive the interaction as game events and triggers. I am not enough of a stickler for words to argue every time someone calls the experience a game. Technically it is not, but it seems the default to refer to any interactive work as a game.

Here is reel I created for the ACCAD motion lab, including some sneak peaks of the project:
 

 
What was great about being immersed in ACCAD (which, as most of you know, stands form Advanced Computing Center for the Arts and Design, the same place I received my MFA from) was the sense of community and being surrounded by smart people who all were doing research in fields closely related to what I do. Whether their focus was Digital Art, Art and Technology, Animation, Interaction Design or, well, Gaming, everybody was willing to share their expertise and help out when we ran into puzzles building our experience that a simple Google search did not solve. And, at least as important, they were there to bounce ideas off of and help develop the conceptual framework of the project. Teaching at a relatively small liberal arts college, I have only a few direct colleagues in fields closely related to mine, and am the lone animator there. From all these conversations with professionally like minded people I definitely learned a lot. And did I mention the great students?

ACCAD Open House. Photo: Lexi Clark-Stilianos

The project, for with he working title Inviting Motion stuck, is complete but not finished. We, that is: Raven and I will adjust some things based on the feedback from the open house (and other instances where people played, I mean: experienced the project) and polish a number of elements. With one last push we should be able to get it ready for festival submission soon!

All progress is slow

Over the past two weeks I have been working on my sabatical project here at ACCAD. Tomorrow we have a big brain storming session planned to hash out the mechanics of the experience. Time to gather my thoughts, time to blog.

For those who do not follow me on Mastordon or Instagram, here is an image from last weeks MoCap session with the talented Ishmael Konney, MFA Dance candidate here at OSU. For next week we have one more session planned with Yukina Sato, also a very talended chaoreographer and MFA dance candidate. I will have three completely different motion sets as the dancers all have rather different backgrounds. Adds an unexpected dimension to the projects, which is exciting and of course a little frightning.

How the piece will open is becomming clearer. The user will be on a platform and based on gaze, using the head rotation a proxy, platforms will appear around the user, with connections growing based, again, on gaze as a proxy for attention. Here is a little 2D test I created trying to figure out how the growing of these connections between platforms, which I refer to as synapses, might work.

While the initial experience flow is taking shape, working out how the generation and harvesting of user data is going to work is very much up in the air. I am thrilled to have an ACCAD student working on this for a class - I have not asked if he minds me using his name publicly. Another student is helping out with the motion captura data. So far, I am not loving Autodesk Motion Builder: ancient interface and weird bugs. Maybe somone should write a competitor, forcing Autodesk to up their game!

So what about AI? The experience should feel as if the underlying algorythm is learning. Which bring us back to where I left off in my last post: If a system act as if it is artificially intelligent, does that make it so?

The topic does not fit this project, but given the crazy warm weather I have been experiencing here, the question whether we may have crossed a tipping point in global warming does seem in urgent need of an answer. But I disgress. Big day tomorrow, better get some sleep.

Contemplating the Conceptual Framework of my Sabbatical project

After an intense first week spend at ACCAD, the Advanced Computing Center for the Arts and Design at The Ohio State University, I am sitting in my hotel room with time to reflect on my process and goals.

The initial concept for my sabbatical project was based on an old idea, namely Ulikeme (or You Like Me) which commented on the pursuit of likes on social media but also reflected on the search for kinship on the internet (are you like me?) Visually the project leans on the test conducted for Ulikeme, but the focus is different. Evolved if you like. I am wondering if it needs yet another update.

The current framework is related to the concept of Surveillance Capitalism, the term for the data mining economic structure from the 2019 book by Shoshana Zuboff. If the service is free you are the product.

Early morning MoCap session with undergraduate dancer Vivian Corey
 

Using motion capture I already acquired great data of human motion. This is one of my goals visiting ACCAD: to learn how to incorporate motion capture into my creative process. The recording was supervised, and co-directed, by Vita Berezina-Blackburn and the motion provided by undergraduate dancer Vivian Corey. For the VR experience I plan to build using this data, I do need to contrast this human motion with other movements. In general I was thinking noise, but what if (some of) the other motion is AI generated?

Inviting Gestures - Midjourney
 

In the ideation phase of the project I started using AI image generators, out of curiosity whether that is a valuable technique, or mainly prove to be a distraction. Though the resulting images were never quite what I was looking for, they did help focus the visual design for the project. Some even inspired me to rethink some of the focus of my project. Did the fact that these AI image generators produced mainly hands when prompted with “inviting gestures” have an advantage over the Google image search also giving me many images of hands? The jury is still out.

When my friend Miho Aoki recently proposed to do a presentation on the use of AI in visual arts courses, I jumped on it and joined her for a presentation we gave for the ACM SIGGRAPH Education Committee’s 2023 Winter SOIREE. The rapid development in AI image generation was already on my radar, this presentation put it front and center. So now I am beginning to wonder if some AI should maybe be included in my sabbatical project. Or suggest AI is being used, give the illusion of an underlying AI. If a system behaves in a way that can not, by a human, be distinguished from AI, is the system then artificially intelligent?

 
Context:
Listening to Thomas Friedman - Thank You for Being Late (2016)
After taking in most of Life 3.0: Being Human in the Age of Artificial Intelligence (2008) by Max Tegmark on my drive to Ohio

Sabbatical

As of yesterday, I am officially on sabbatical. As part of my Spring '23 sabbatical, I will be spending two months in Columbus, Ohio, as a Visiting Fellow at the Advanced Computing Center for the Arts and Design (ACCAD) at The Ohio State University. I wil be working on an interactive Virtual Reality (VR) experience on the theme of enticing and luring, attention grabbing, and having to give something up when we give in to the attraction. More info on my website

Data Suckers, concept art, 2022