Warhorse – The Rip

Posted on December 13, 2024 by andy

Late august my good friend Jack Nicklin-Jewell got in touch, he was in rehearsals  programming the upcoming tour of warhorse and had some issues with the 3D model of the projection screen didn’t match the physically built screen.  d3 was being used as the media server of choice for the tour and it was important that the mesh used in d3 was matching as close as possible to the physical set

Jack had been supplied with a cad of the drawings that the set had been constructed of but additional set dressing had been added to the original plywood base to make it look like a giant piece of paper torn from a page, thus why it is called The Rip. The install of the show in various locations around the UK was going to be very tight and handled by LX,  to help with a faster slick lineup Fibre optic LED were placed embedded into the surface of the screen. This meant that for lineup it was a simple task of dragging the “quickcal” points from the 3d mesh  onto the physical set and the software would workout the rest.

When I first got the call, the production had requested a scan with my Faro Focus but on further discussion and finding out what the actual job was I decided that reconstructing the set using photogrammetry. The main reason being was to make sure the fibre optic lineup dots were captured, this is a detail that could be missed by the the scanner and also I have more control of the actual image texture captured in terms of light levels and quality of image VS the image capture on the scanner.

I was already traveling down to london to do my yearly CVC banking conference with Graymatter and Giggle Studio so I also packed my Sony A7V to go and do the image capture of “The Rip” at Wimbledon before my main job. Over the last few years I’ve been using photogrammetry quite a bit for constructing models but only in the last 12 months has it became part of my official workflow supplementing or replacing a traditional scan depending on the project and the environmental factors.

I arrived in London early and checked into my hotel, I wasn’t on the Graymatter job until the following day so this fitted in nicely. I packed my camera kit and headed over to the New Wimbledon Theatre where they were rehearsing warhorse. I met Jack & Alex there and we went in to find an opportune time to capture the images, I needed a clear stage but as soon as the actors had finished onstage everyone else wanted access to the stage to do other stuff, classic theatre. We eventually found a time where i had 20 mins to get the images, I had already prepared ahead of time some 4k static noise to project onto the surface, this was done to generate some features on the surface of the plain set piece to help the software reconstruct a mesh. Without any features, or very few its very difficult for the photogrammetry software to generate a model form the images, projecting a basic static pattern on a plain surface to help the software was a little trick I developed when doing some R&D into this problem earlier in the year and it works a treat.

Above are a few of the R&D images using projected noise to create a reference base.

Eventually after rehearsals were done i had time onstage to capture the images, it was quite a dark stage even with the lighting and projected noise hitting the projected surface so had to use a higher iso and shutter sped thank id have liked to but i was on a tripod so the shutter speed wasn’t too much of a problem. I took several test shots, checked them on the camera, adjusting my camera values until I was happy with the settings. I think I took around 80 images in a linear parade at various distances and angles to the surface. In addition to the set piece I also made sure I captured the proscenium arch which i had physically measured the distance and then I used this distance to scale the whole scene when reconstructed so the virtual matched the physical.

I brought all of the images into Reality Capture after a little bit of tweaking in DXO photolab and RC did an excellent job of aligning the cameras straight away. I did a reconstruction box to get rid of the rest of the theater that didn’t need constructing but kept in the proscenium left and right which i was using to uniform scale the scene. The trick with projecting noise on the surface worked a treat and it gave me a nice clean construction of the surface.

Once I scaled the scene, I filtered out all of the polys that weren’t part of the set piece, generated a 8k texture and then exported it as an FBX ready to go into Cinema4d to do the retopology and UV.

There was an existing model built from the cad that didn’t match the set butt here was also the associated UV with the model that I needed to closely match with the new retopo I was doing from the capture data. For testing purposes to send to to jack to make sure the workflow fully worked I did a bit more clean up of the mesh in C4d and did a quick mesh using Quadremesher a great tool for a quick quad based mesh. I then did some further clean-up on the re-mesh, matched the UV and sent it over to Jack for testing.

I then went back to the model and started doing a much cleaner retopo, quadremesher is good but in many cases doesn’t give a clean edge on an object of this shape. For my manual retopo I started with a cylinder shape and scaled it up so the radius of the cylinder sat exactly un the “cup” of the screen mesh. I then messed around with the horizontal and vertical sub-divisions to get a good density of quads, once I was happy I made the object editable, roughly deleted all the quads I didn’t need around the edges. Once I had the overall shape, I used the photogrammetry mesh as a guide and using the knife tool I cut around my really clean quad mesh until it was exactly the same shape as the capture mesh. Its totally worth doing these things manually so you have full control of the process, also having a clean mesh like this made the UVing so much easier and cleaner to closely match the existing UV

The next step was to locate the fibre optics from the photogrammetry texture and transfer them to a UV layer so this could be overlayed on the mesh for line-up quck-cal the projection during line up. For this is basically overlayed the source mesh with texture on top of the new mesh and then using the 3d painting mode in the UV window in c4d I painted directly in the new mesh and this dot transferred to a UV layer. Below is a basic example, using where I painted a face on the cube in 3d space and it transfers tot he UV, its super useful for things like this.

In addition, I made another mesh which matched the fibre optic markers as a helper mesh which could be loaded into d3 in addition to the clean mesh as that’s how Jack had it setup and I like consistency

That’s it for this blog post, this was fun project spanning multiple skillsets and also working with good people.

FIN

Posted in cinema4d, d3, Photogrammetry, Projection Mapping | Leave a comment

EWC with Bright Studios

Posted on October 18, 2024 by andy

June through to July this year (2024) I was out in Riyadh, Saudi with Bright doing the E-sports world cup, previously known as Gamers8. This was the third year doing this with Bright and the gig has grown considerably since year one which feels like so long ago now.

Some of the year 1 team

The event revolves around some of the best esports teams in the world who have been through various stages to get to compete in the world cup in Riyadh. A multitude of games are played over a course of several weeks culminating in a grand final based between 2 teams with the most cumulative points.

2022 Games arena

Year one we had one main games arena for the playing of the games and one studio for commentary during and between the games, all if this being streamed to various online platforms mainly twitch and youtube.

Both the studio and the arena had multiple AR elements to the show to have game characters appear onstage in camera next to the players and also a very grand opening sequence which started the games each day.

Tabea & Matt Calibrating Camera Lens’s

With two and a bit weeks setup before the the first event there was a lot to do, Stageprecision was used throughout, a team spent about a week calibrating the camera lens for all postion’s creating a lens profile to match the exact camera body it was attached too. This was then used to take into account the lens distortion when overlaying the AR which came directly form Unreal via the Stage precision Shield plugin.

SP Dashboard

In addition to controlling all of the Unreal nodes for the AR, stage precision was taking in all of the game logic API and based on events in the game it was then sending triggers audio lighting and disguise. This meant when there was a “spawn” or a “kill” event in the game LX Video and Audio would be triggered at the right time for the correct look or sound stem.

Krystoph and Tabea measuring for the camera nodal point

And another thing SP was handling was the tracking data coming from the cameras, reformatting and offsetting the data to get the physical and the virtual worlds aligned so the AR looked correct. All of the cameras had a Stype Red spy system on, apart from the spider cam which had its own system. We had quite a few issues with the spider cam data drifting and needing resetting quite often which was a bit trial and error to start with but once everyone got in the swing of “how to make it look right” we were golden. The Spider cam tracking looks like it works great when you see it on a stadium size gig but for our purposes it didn’t really cut it for the accuracy we needed in this smaller size venue (more on this later).

This was the control position for year 1 in the same arena as the games were being played, we were behind the seating bank and as with most gigs in Saudi it was a building site for the first couple of weeks. Everything was continuously covered in dust and the temperature swung between hot and cold as the people who were in charge of the building tried to get the correct air-con setting.

I was operating d3 on the gig which was delivering all the content to the led and rendering scores to the led via a notch block running over renderstream from RX servers. The notch block got the games score data from the API via SP and we also got indirections from the games engine API via SP which pulled up the correct player/team name before going to a particular cue. All of this was triggered by a button push on a streamdeck that sent a command to SP which then pulled infor from the games API and sent the relevant triggers to multiple departments.The whole system integration was complex and had a lot of moving parts, the above is just a few. And to those who are reading this who got lost in the terminology heres a breakdown.

d3 : disguise media server
Indirection : The terminology when send sending a file path string to d3 to display content in a video layer/module
Notch : realtime rendering engine that can be played and procedurally updated within d3
Renderstream : disguise’s proprietary system for sending a video stream over a network
RX : Rendering machine part of the disguise ecosystem
SP : Stage Precision live event and production toolkit
API : Application Programming Interface, a way to pick up data from a computer program over a network.

Everything worked smoothly except renderstream which would crash and bring down the servers after running for around 5 to 6 hours and most of our broadcast was around 8 hours each day. The disguise devs were great and we spent a long time with them trying to identify the issue but it was never resolved. In the end we moved the notch block to run on one of the local Gx machines which worked a treat.

Heres one of Krzystophs brilliant memes which helped keep us going

That was a brief recap on year 1 of Gamers8 now we fast forward to 2024 where the event has grown considerably from 1 arena and 1 studio, in 2024 we had 3 x arenas, 3 x English studio’s and 3 x Arabic studios. Usually I’m out for the setup for this but this year Matt & Pillerin were holding their wedding party which Kristaps & myself were attending. This meant Kristaps, Matt & myself didn’t go out until 10 days into the setup which as were all kind pleased about. The celebration was really nice, we all met in Tallin, Estonia and enjoyed a great party and generally just hanging out

Happy smiley faces!

After flying home from the party, repacking and sorting out the flat I was on a flight the following day to Riyadh, with a quick stop in LHR where I met Adam for the next flight and we arrived in Riyadh late that evening. The following day we all met the team who had been here for the last 10 days and heard all the horror stories of the site not being finished which still didn’t really prepare us for when we got onsite and seen ourselves how still unfinished it was.

The building was sill being built when we got there, windows were missing, the toilets were still being constructed and the air-con didn’t work. It was at this point i actually realised the extent of the shitty environment the guys had to endure for the first couple of weeks before we got there and they had done a great job managing to get everything in.

View to the left of my control station with Ufuk working on one of the studios.

In 2023 all the operating positions for each stage were in the same arena that the stage was in, either operating from behind the set or behind the bleachers. This year we had a unified control room with the servers at the stage and all remote KVM to the operating positions. Being in one space had its pros and cons, it was nice that it was quite social being able to see each other and bounce ideas between us. The down side was that the space was just too cramped and got quite noisy, nice idea, just needed more space.

View to my right featuring Nate and Kris

For the first week we were onsite for one day, off for three because the content wasn’t read then back in for the remaining 3 days for some frantic programming and collectively figuring out how the show should opperate and fit together with all the other departments. I think it was on the 4th f July we were meant to have our first full rehersal with all departments but that turned into a slow stumble through. Everything was delayed generally hampered by the building flaws the general struggle to get things done in the Saudi environment

Screengrabs to verify Triggers

Rehearsal’s, once we got into the swing of them were supper slow, there had been constant changes between different departments and quite often early on these changes weren’t relayed to everyone. This mean that when I hit that button on the stream desk it didn’t always turn out as expected. Sometimes the wrong states/character’s were being displayed and I spent quite a bit of time on coms with he various departments figuring out if we were sending he wrong command or they had programmed the wrong “pickup” at their end.

Eventually we were all lined up across departments and things were working as expected, the AR looked great so crisp and the tracking was really tight. I was there for only 2 weeks of games which was enough. All of the interesting and challenging stuff happens in the install and set and after that I just became a button pushing monkey with a bit of managing communication between departments.

Cube Stage control, my work view for 3 weeks.

It was a good few weeks working with friends, making new friends and hanging out in an dusty parking lot between construction equipment waiting for Ubers.

Rury doing the where’s my uber dance


Posted in d3 | Tagged , , , , , , , , | Leave a comment

The Eurovision Post

Posted on August 1, 2024 by andy

It must have been around February 2023 when I got a call from Chris asking if I was up for being part of the video programming team for the Eurovision Song Contest 2023 in Liverpool. I of course said yes, not only because it was an exciting gig to be involved with but more-so I knew who else was on the team and it was a bunch of folk I love working with.


This year the UK was hosting Eurovision on behalf of the UK and the venue was Liverpool arena. For the event the spaces in the arena that were usually used as offices for the day to day running of the complex were cleared and kindly handed over to Eurovision for the duration of the gig.

Harry(Hazza)R, DanH, AlexP, DanL and MikeS from QED were onsite with us doing a sterling job of getting the system in and keeping it up running. We had a boatload of disguise servers for the massive led screens, several projection elements for the show and also Luke, Jack, Sam & myself had a dedicated server each as an editor for system consistency. Ed was also on the team and in charge of media wrangling for all the content that was coming in from various providers.

Dan & Harry had set up a nice system for each of our programming stations where we had a stream deck triggering presets in a rack-mount audio mixer which was routing the different timecode sources and audio to and from the editor machines. This meant we could each be independently programming parts of the show using the shared audio tracks striped with the relevant timecode played back from our local machines. It was a really nice clean solution and helped speed up the workflow being able to instantly switch and listen to TC and audio sources so quickly.

Also part of the team we had Emily, Brigsy and Max from d3 who were onsite with us for support and implementing the spike mark system for the event. The spike mark system was a nice elegant solution where the stage management team who were onstage with ipads were able to notate positions on the led drawing various symbols. These could be saved as snapshots/presets pushed on the server sending signal to the floor. Emily and Brigsy had developed the main core months previous to us loading in and were constantly adding and improving to the system onsite. The first time we had a try of the system onsite it was really intuitive and we (I) came up with this.

I wasn’t the only one drawing speed dobbers onstage, here are team d3 looking proud, apart from max who isn’t as childish as the rest of us.

The d3 programming part of the show was pretty smooth and we had enough people to handle the work during rush times so there was no real stress moments. When content came in we divided the work to program individual tracks, sync the timecode tags and check for errors. Most act content came in as single tracks but there were a few that needed sequencing and some with camera perspective tricks that needed care and attention working with the director and cameras to get it looking correct. Luke was lead programmer pulling in the individual programmed tracks between us, assembling the final show timeline and sitting in the hot seat for the show.

In addition to the content on screen for the competition performances we also had content fir the presenter looks, the flag parade and the interstitial performances which happened between some of the songs. For this we had a team from the lovely people at Northouse who are always great to work with. There was quite a bit of sequencing i needed to do for this as there were many requests for changes in the content from above so we were receiving small sections at a time and adjusting timings as and when the audio tracks changed.

Try and find a picture of us all with northouse to go here.

I always try to create a nice working environment when on jobs and im a big fan of having ambient lighting around the office, so during the first few days i distributed fairy lights around the team to decorate the desks and also bought a few lava lamps for the mood too. Chris joined in on the fun too with his northern lights ceiling projector and a scent machine to make the office smell like a cheap spa.

We had 2 projection gags during the show, one for Ukraine and one for Armenia. The ukrane set was wheeled onto stage and the alignment had to be spot on to avoid any spill and also it was tightly mapped to the choreography. After a few teething problems hetting the set on and off and set in exactly the right position the stage mangent team were really good at hitting the mark with the set.

Even though the stage team got the location of the set spot on there was always room for tweaking and during to the show Jack was tweaking the lineup via a camera shot in the short turnaround before the number.

The other projection gag for Armenia was a trapezoidal’ish shaped ramp that the artist was standing on, in the projection. For this the set was made up of 2 pieces and the crew wheeled it onstage, stayed under it during the performance and wheeled it back off, it was brilliant to watch.

Heres an image of harry on the ramp being a stand in whilst the projection was beign focused and tested.

We had a 50k firing at from around 80m away at the set piece and it was so bright with no problem of being washed out by lighting.

In addition to the projection gags we also had a couple of forced perspective gags for an interstitial performance by Ukraine and content from Freckled Sky, the other was for the UK entry. The Ukraine pwerformance calles welcome to our house was a beautiful piece with two dancers performing against led backdrop depicting what was going on in the Ukraine at that time. There were several tricks in the content and choreography where through the camera when the performers walked behind a slab of led it looked like they continued into a virtual world as the content carries on their movement and silhouette.

The content was rendered out from a “hero shot” perspective where we had to map onto the virtual set in d3 and then this was displayed on the physical set. We then also had to work with the director and camera department to match as closely as possible the physical camera to our virtual camera, position and focal length.

Below is an embedded video of the performance which helps visually show whats going on much better than I can describe.

Id never really watch Eurovision for many years, I do remember watching it as a kid but this was the first time in ages that I was actually mentally invested in the competition. Between all of us there were 2 clear favourites that we were routing for and those were Croatia & Finland. Outside the venue Croatia had a mobile sauna and one of those good photo opportunity which we didnt think twice about sticking our heads through the holes for the camera.

After the show was over all of the acts were walking past our control room and everyone was cheering them on. Alum, one of the team from CT was a fellow Croatian of the band Let 3 and he fagged them down in the corridor and got a picture with them.

Overall the event was really well put together and such a wealth of talent working on a show together, it was one of those gigs that you kept bumping into people that you’ve not seen in a long time. This post could be much longer going into the technical intricacies for the show, how the whole team where referencing CuePilot for the programming of the show which is a fantastic system, to how we were picking up the automation and recording it into d3 so we had the ability to program offline with the correct screen moves. Its been nice to write this but have left out a lot, if theres anything yous like to know about the screens side please drop me an email.
Of course not forgetting the whole gig was peppered with googly eyes being stuck all over the venue and Liverpool.

Posted in d3, LED, music, Projection Mapping, Work | Tagged , , | Leave a comment

Touchdesigner Animation Subroutine

Posted on July 13, 2024 by andy

Hello!!! Week three here in Saudi working for Bright on the Esports World Cup and we’re settled into a routine with some time to be doing R&D for future projects. This week I’ve been doing a bit more discovering with Touchdesigner trying to figure out how to trigger an animation event. After few hours searching experimenting I found the animation operator which is fantastic tool allowing you to keyframe values which can be fed into assets to keyframe movement. Bellow is my simple patch which can be packaged up into a container and duplicated and used multiple times with different keyframes animations within a single project .

The heart of it is the animation COMP which doesn’t look like much until you click the edit button in the properties window and low and behold a window appears that allows you to add keyframes / edit them and overlay them on-top of each other for useful visual programming.

Adding channels is simple and can be tailored for each project, key framing is a little bit clunky to start with but there is a useful help window if you right click in the keyframe editor. After 10 and of figuring how to use the tool, key framing becomes quite intuitive and you can fine tune each key with a time and value above the timeline. Below is a simple animation I made key framing the rotation plus the x/y scale over 250 frames.

By default the animation op is locked to the project timeline and I wanted to be able to trigger and retriever the animation on cue. My solution to this was to use the counter op and set it to Increase Count in the While Off field. This will continually count up on each tick unless you set the limit to Clamp Min/Max. I set the Max to the length of my animation and I then used a keyboard input to trigger the reset, In addition I added a reset pinpoint value to be able to trigger further into the timeline easily. Below outlines the basic process and its a nice easy network to setup.

Historically I’ve used vvvv to program systems but since last year Ive been transitioning my knowledge over to Touchdesigner re-learning how to do things which I knew like the back of my hand in vvvv. Its important to keep learning new stuff, even of you don’t ever use it in the future its nice to play. That the end of this little post and to finish it off here’s a tenuous face int my banana at work.

Posted in Touchdesigner, Uncategorized | Tagged , , , , , | Leave a comment

E-Sports in the dessert

Posted on June 30, 2024 by andy

Its the end of June 2024 and I’m sat in a hotel lobby in Riyadh with a couple of hours to spare before I need to head into work with the rest of the d3 programming team. The week started by flying form Newcastle to Heathrow on Monday where i bumped into Adam who was also flying out to Riyadh to work on the same job.

We had a couple of hours to kill and had a lovely catchup over some very spicy bloody mary and some pretty good lounge food. The flight was good, we were stuck on the tarmac for a couple of hours before takeoff due to air traffic and i managed to watch the film poor things before being airborne, the film was really good, I highly recommend it. We eventually got to the hotel around 02:00 in the morning and was up agin for breakfast at 08:00. It was nice to catch up with people id not seen in ages and hear how thing were progressing onsite, everyone said the whole place was still a building site and they weren’t exaggerating.

The control room was looking good, many of the guys had been here for over a week already and had done a great job. There was still grinding works going on in the building and everyone was working around each other.

l

This is the third year Ive worked with Bright Studios on this esports event and it evolved considerably each year. Its grown from 1 games arena and 2 studios to currently 3 games arena and 6 studio. Previous years the operating positons were all at the respective stages so we were all spread out, this year for the first time we are all operating from the same area remoting into the servers at the relevant stages. The first thing I did when I arrived was find my desk and figre out how i was going to arrange things to make it a comfortable and an efficient place to work at for the next few weeks, this invoived going round site, scavanging some wood and asking the chippies to cut it into sections so i cold build a small shelf on the desk to make more space

Shelf image here

Ince that was all done with i settled dow and got familiar with the system, it was nice to be sat next to Kris and Dan, and bounce ideas off each other with the initial round of programming.

The event which we are working on is called the the Esports World Cup previously called Gamers8 and seems to be growing year on year. From what I understand there is a huge chain approvals needed for all of the assets and content and because of this, much of the content needed to be re-rendered as there was so many changes. For the team of us programming this meant there was limited things we could be working on and we completed all we could on the first day.

Several of us were told we didn’t need to come into work the following day and were given the day off so decided to do some planning on what to do with a whole day free in a city were the main entertainment activities are going to a shopping mall or eating shawarma. So thinking of ideas on what to do, the following day Nick, Robin, Carsten and Myself decided to go to a mall and the first one we got an uber too was closed and looked quite derelict inside.

We eventually went to a more lively mall, and did a bit of mooching around and grabbed some food. The rest of the day was spent by the pool wondering if we were going to have any content to program with the following day. I think it was late afternoon/evening we got the message that there was still no content ready and no need to come into work the following day! With that news I suggested we hire a car and drive to a place id been told about called “The Edge of the World” which is a tourist spot 2 hours outside of Riyadh with magnificent views, with this suggestion, an adventure was brewing.

Little over 12 hours later, 6 of us were at the airport picking up a Chevy Tahoe hire car ready to drive to the edge of the world, we knew the “easiest” route by some instructions from the internet. After a quick pitstop via the supermarket for plenty of water, snacks and general supplies we were headed off towards Sadus Dam, this was allegedly the “small car route”

The first part of the trip was pretty straight forward, leaving Riyadh in tarmac roads all the way to Sadus dam via some nice changing landscapes.Once we got to the dam the satnav took us directly across the bed of the dried up dam and west toward the edge of the world. At this point it started to get interesting as the road was becoming more and more rugged, Brigsy and Carsten were sat right in the rear seats over the back axel and were getting bounced around all over the place.x

The road continued like this for about 30 mins, some rough bits and some smooth and it was reassuring that we had a big land cruiser to get us around. We did encounter a couple of chunky obstacles which took a bit of figuring out if we were gonna clear the ridges but Dan and Nick who shared the driving did a sterling job of navigating over them

After arrond an hour of offroad driving we made it to the lookout pont, we could only drive so far and had to walk for about 10mins in. the baking heat to get to the vantage points. We were the only people there when we arrived and it was pretty amazing to have the place to ourselves.

And a selection of photos from Robin Verlinde with his new Sony camera

On the way back we weren’t sure the truck would make it up the flatter hill we went down when going out so we decided to try going over the big berm, when i say we i mean, we collectively suggested it and then Nick had the balls to try it.

We had bought all the snacks to have a picnic in the desert but none of us fancied traveling over this terrain in the dark so we decided to aim to get back to Sadus dam before sundown and have the bread cheese hummus and meats there.

Thats the end of this weeks blog post, its been a great week catching up with old friends and meeting new ones. Theres still a bit of a delay with various knock on things on the production and I’m sure were going to have a couple of mental days next week during rehearsal’s once we get everything.

Until next time, heres a Face

Posted in Uncategorized | Leave a comment

2024 so far…….

Posted on June 9, 2024 by andy

Its almost June already, the first half of 2024 has flown over so quickly so heres a quick recap of whats been going on so far. After the joy awards 2024 I had a week or so at home and then it was time for the yearly snowboarding trip. There was 8 of us going out his year to La Clusaz, James, Charlie and myself form the UK, Ruben and Fernando from Spain and finally Matt Pillerin and their 1 year old Tilda from Estonia.

We all met in La Clusaz to be greeted by some incredibly hot weather and immediately thinking that the snow wasn’t going to last very long at this rate. Luckly up on the slopes it was still cold enough and we all spent a great week on the slopes during the day and eating drinking and playing board games on the evening.

We managed to get sa few black runs in this year without any injury and this is Matt, Ruben, Fernando & Myself at Massif de Balme where despite the heat in the town we always had plenty of snow on the top of the mountain. As always it was a great week in La Cluzas well spent with friends.

Straight after snowboarding I was headed to ProductionPark at Wakefield to help Pete Tilling program the 2024 Niall Horan tour. We had Caspar and Jos from SHOP doing content who I believe we had met on email, on another job and nice to finally meet in person.

I’d done some pre production for the moving Austrian screen and the general project setup but onsite there was some quite heavy lighting integration where we were sending lots of video to DMX from table screens in d3, mainly the light box above the stage and we tried sending to some of the movers upstage but that part was cut as it just didn’t work.

For over a year now each time I’m programming a show and in rehersals mode I put some animated googly eyes up which I made in January 2023 on one of the Joy awards and they have became a staple programming asset since then.
During Niall programming the eyes were up on the border between rehearsing songs, as we got closer to final rehearsals Ben, the creative director told us he, and Niall would like to keep the googly eyes in the show.

I found it so funny and quite pleased that they wanted to keep the eyes which come up on the last song just before the band come on, and below is me at the first show in Belfast late Feb 2024.

Shortly after returning from Belfast I was down London with Graymatter for a ITV showcase/premier thing at Leicester Square Odeon cinema. It was nice to see Dan from Graymatter and also Dave & Toby from Hawthorn who I didn’t know were going to be there.

The operating position was at the very back row of seats in the cinema and was pretty cosy, “Smile for the camera Dan!’

Another fun thing which happened in February was the d3 “after dark” event aimed to help the company reconnect with the user community. It was great to meet old friends and put faces to names of new friends. The gathering started at d3 hq at southwark and then we splintered off to a few pubs. After the event it was even more unclear which direction the company disguise is heading but everyone had a good time chatting shit and trying to figure it out.

Last pub of the evening with these lovely people.

There’s has been many other things and jobs that have happened this year but unfortunately for the blog, so many projects are under NDA these days so can’t write about this at the moment.
After the d3 get together my next job was with the lovely people at Pixway with Paul & Ron in Amsterdam. This was a big dealer conference presenting the new line up from BMW for next 18 month. The whole event was very locked down and we all had to deposit our phones in a locker prior to going into the venue for work. this made communication quite difficult too as very few people were given radio coms.
I was programming the evening show and I definitely got the easy end of the job as Paul & Ron had to deal with late content which is never fun.

It was a great week and lovely to catch up with Paul & Ron, at one point Ron and myself had both brought a couple of packets of troop waffles to work and we did collectively and up eating them all before the end of the day.
I think that’s enough of an update so far, it’s kinda unbelievable that we are halfway through 2024 already and it feels like its just begun. here’s to a just as enjoyable second half to the year.

Posted in Uncategorized | Tagged , , , , , | Leave a comment

Three Years of Joy

Posted on March 19, 2024 by andy

Here we are the third month into 2024 and it feels like this weeks have passed by so quickly, I was going to start updating this in January but as so often these I’m not so sure where the time has gone so here we are at March 15th.

For the past 2 years I’ve assisted OgleHog/Chris Saunders with a big event in the middle east called the Joy Awards which is basically the middle east’s Film and TV awards show hosted in Riyadh.

Below is a breakdown of the first one we did back in 2022, produced by Done and Dusted with set design by Stufish

Stage screens breakdown in c4d

Tubes projection analysing with my c4d projector plugin

Tech’ing onsite

Its evolved in the past 3 years since January 2022 where we were a team  of 4 of us, Dan Gray, Toby Vogel myself and supported by Creative Technology’s Carlos Aguilar. That first show was a bit “wild west” there was so much going on with many led surfaces and also projection, there was so may cues to program and the show running order and content was changing constantly. We all worked hard for our money on that gig, it was great teamwork and we pulled of a totally clean show.

2022 show opening shot with projection on gauze & tubes plus loads of led

Fast forward to November 2023 and Chris called to see if I was up for doing it again and begin the video programming planning stages. There were much fewer LED surfaces this time but exponentially more projection gags with projection for almost every act and basically wherever else it could be squeezed in.
Below is a screengrab from my cinema working, showing the extent of the set which was beign built to augment the auditorium.

Thus years design incorporated more auditorium dressing than previous years which all was to be projected on. Over the next few weeks based on the Rhino3d from Stufish I came up with a projection design for the auditorium with some great input form Dan as he was ultimately lining this thing up onsite.

Mapping Matter early projection study

The other projection elements were very much influx with in the whole run up so prior to the equipment shipping date we decided to spec a few more projectors than we needed and some extra lens options. This meant we had enough kit to make stuff work on the constantly changing landscape of the Joy Awards 2024. Doing something like this in the UK would be much more straight forward as if you were short of kit, then its relatively easy to get extra kit sent to the gig, this is totally not the case in Riyadh at the moment.

Chris Harry & Ben pondering the days events

On the 7th January a team of us flew out to Riyadh for the setup at Riyadh BLVD (Boulevard) which is a bit of a cross between a theme park and a shopping mall, its pretty much the only theatre space I know in Riyadh. Pete Tilling was out a few days earlier to receive the kit and get the projectors the basic locations and was nice to see his smiling face when we arrived.

Just before getting told off for siting on the sculpture

The following few days went really quickly with the guys from QED getting the signal in, rigging and pointing the projectors. Dan and Harry did a lovely job of the server cabin which was a super tidy install. In there programming cabin there was 4 of us on the buttons, Dan Gray, Ed Joynson, Adam Power and myself. Ed and Dan eventually ended up on the late shift lining up the projection in that lovely time of evening when there is no-one else there.

The “fins” being installed

Dan Ed & Pete

Dan lining up and sorting blends

This year, I brought my laser scanner out after a bit of trouble last year where the physical set was vastly different from the 3d design, which meant Dan spent a lot of time moving 3d elements and warping in d3 to make it fit. So first thing this year once the auditorium set resting was in and complete, Dan took the Faro s70 and scanned it to create an as built point cloud, and then use that to modify the existing model to the real world setup.

With the advantage of the scan data we managed to get the map the roof pretty accurately with some sever projection angles too, it actually looked better than I was expecting with the focal distances we were covering.

This was my programming setup, Dan Hall did some fantastic companion work patching my Stream Deck Plus to the Behringer rack mixer so I could switch timecode from local source to program source and also control the incoming program audio feed with simple button pushes.

It was great to have a bigger team this year with all of the onsite challenges we had to deal with, everyone was so capable and a Joy (excuse the pun)to work with.
I was looking for images of the actual event but I didn’t take any of it complete. The final show went well and was also super long (almost 5 hours) and it was another fun challenge following along as 98% of the show was in Arabic.
Lets see if were back together for another adventure next year.

THE TEAM
Lead Progammer : Me
Programmer : Adam Power
Programmer/Lineup : Dan Gray
Programmer/Lineup : Ed Joynson
Technical glue for video : Pete Tilling
Screens Production : Chris Saunders
QED Team :
Harry Ricardo
Dan hall
Paul Morland
Ben Shepard
Toby McManus
Alex Peggington
Steve Hardcastle

Parting our ways returning home at LHR

Posted in Uncategorized | Tagged , , , , , , | Leave a comment

Nike Air Max Day 2023

Posted on April 25, 2023 by andy

Early march I got a phone call from Pod Bluman of Bluman Associate’s asking if it was up for being involved in a projection mapping project for Nike Air Max day 2023. On the project already was Rich Porter looking after the projection study + project setup and also Lewis Kyle White looking after texturing and content on the new giant Nike Air Max were were to be helping launch for the agency Amplify.

The giant shoe we were projecting onto was to be situated on top of a car park in Peckham opposite the venue where the launch party for the new Nike Air Max was to be hosted. Due to the ovation of the shoe and the forecast high winds my involvement was to help track the shoe if it was buffeted by the wind so the projection would stick to it. This was because the shoe was hung on steel cables and due to the nature of the rigging it was impossible to keep 100 percent static.

Lewis made a great job of remodelling the shoe from high poly mesh into a low poly mesh ready to be sent to Stage One to be machined and built in real life measuring and actual size of 10m toe to heel. The whole thing was suspended in the middle of a 17m cube of truss and supported by heavy steel cables. Above is the incredible texturing work Lewis put into the modelling, the fabric looked so realistic when projected on the shoe.

In the pre planning stages the guys from bright studios helped out consulting in the best methodology for tracking. In the end we all agreed that using Optitrack to track the shoe & Stage Precision to format the data in a useful way for the disguise media server to receive it was the most flexible solution

Bright provided us with an 8 camera tracking system and prior to going onsite we had two days pre production in the Bluman studio space to do some testing and work out the technicalities before going onsite. It was great to see everyone who id not seen in ages and also be joined by Ed who was programming video for the launch party across the road from the carpark.

When I got the the studio space and after having a good natter with everyone who I’d not seen in ages I set up some basic rigging for the cameras. We were meant to have a 1m 3d print of the actual shoe to test but we couldn’t get one done in time, so instead Lewis decided to hang the sample shoe from the truss for the craic.

We had a great couple of days testing and working out a workflow for getting the tracking data from Motive to disguise using stage precision in the middle to align the tracking data and the virtual world. Over the weekend the team worked getting the truss and the shoe rigged ready to get the projection in and lined up from Monday.

View of the shoe from the party venue over the other side of the railway tracks from the carpark which houses Peckham Levels. The first thing I had to do once getting onsite was to attach the tracking markers to the shoe, these are small 10mm retro reflective markers. The tracking cameras have a built in light which blasts out infrared and the markers reflect the infrared light back to the camera. Using multiple cameras looking at these markers 3d map of the markers position is formed.

I covered the lower part of the sole of the shoe which could be reached with a small cherry picker with seven markers. This was enough for the camera’s to pick up a large enough area and detect the movement of the shoe.

This below is a reconstruction of the markers attached to the shoe in the Motive software. Due to the nature of the system using infrared light we couldn’t calibrate the system or check if the cameras could pick up the markers until sundown. For a couple of the cameras we were close to the maximum distance limit for the system but as we did the first calibration of the system they picked up pretty well. I ended up moving some cameras around from the original plan to get better coverage, and after some settings tweaks we had the tracking data come in really well.

Once I had the tracking data set up I switched on streaming in Motive so I could pick up the NatNet data in stage precision. I was going to try and use the world align tools in SP to align the tracked points to the fixed points to where the projection on the shoe was calibrated too but this didn’t quite work with this setup and I’m told this is fixed in the current version. Instead I manually did the XYZ offsets to match the tracking data to the static position of the shoe which worked a trick, after all the offsets were correct i passed the tracking data to disguise as a FreeD tracking data packet which Rich attached to the shoe object. The whole process of lining up projection then offsetting the data relied on weather being good as the shoe had to be static for the first initialisation for this for it to work, luckily we had enough moments of good weather for this to happen.

In the end everything came together and as the shoe was buffeted by the wind the tracked data passed to the disguise project and helped the projection stick to the shoe, a great success. In addition to that it was great to catch up with a load of people id not seen in ages and have a really good time in Peckham over a few beers after work.

Posted in Uncategorized | Tagged , , , , , , , , | Leave a comment

Come Home Again

Posted on December 2, 2022 by andy

Earlier this year I was asked by Luke Halls Studio to work with the team on a project by Es Devlin called come home again. The piece / sculpture is designed to bring awareness of 243 at risk species of nature and wildlife throughout the UK.
I was brought onboard to look after the work flow to projection map onto the sculpture from generating the UV templates for the animators through to laser scanning the sculpture and remodeling elements to enable accurate projection lineup.

Sculpture during the day

The sculpture was built by Stageone based on the architecture of the dome of St. Paul’s cathedral and was adorned with cutout drawings of the 243 species all had drawn by Es. In addition to the projection element there was also a choir performing with a sound scape in addition to Es taking about the species a la Marks and Spencer style advert. And lastly there was a lighting element thrown in for good measure to backlight the decoupage style set pieces.

The whole process started with myself receiving a Rhino3d model of the sculpture and having a chat with Charlie who was leading the video design on this to figure out how to breakdown the model and UV it so the content templates would be flexible and also easy to decipher.
The model as split into constituent parts consisting of the Base model to which all the species were mounted and each species split out, these were, lichen, birds, insects, mammals, plants and fungi. Each species was given its own UV map which when all collected together created the map for the whole projection mapping of the physical structure.

Some of the multiple UV for content creation

The rhino project was pretty busy, not only did it have all the 3d bit that we needed it also included every single nut, bolt, mounting bracket, EVERYTHING. This took quite a bit of cleanup to split out the parts we needed and the parts we didn’t need and reduce the model down to few enough polys so the media server software could run smoothly. This was a really long process and I’m still trying to find more streamlined ways to make this quicker. It was only after isolating and grouping the species and the structure that I could generate the above UV maps from the cleaned up 3D geometry.

C4D model generated from the cleaned up rhino file

At this point we had two versions of the sculpture, one was the virtual 3d model and the other was the as built real world structure. In my experience, temporary structures of this scale and complexity are very rarely the same as the drawings they were built from. For this exact reason, a few years ago, I invested in a laser scanner (Faro s70) to specialise in laser scanning for projection mapping. Once the structure was finished I got my friend and colleague Paul to come down with the scanner to get an accurate 3d scan of the as-built structure.

Wide view of the scan data captured

The laser scanner basically takes millions of measurements from a point and builds up an accurate representation of the surroundings called a point cloud. Once the data has been captured and processed into a point cloud viewing software of choice you can move the virtual camera around the 3d scene and view the capture from any perspective which really pleasing to look at. Below you can see the limits of the capture which is 70m so that’s why some of the background elements disappear into the ether, anything outside of a 70m radius of the capture is lost.

Closer view of the point cloud data from the scan

From the point cloud data I spent quite a bit of time cleaning this up so I only had the data of the structure and the animal cut-outs to then turn into a mesh for use in Cinema 4d. From the real world scan and the existing model I could then overlay the scan data on top and see how closely things matched.

Clean mesh being placed on top of the scan mesh

As expected the CNC cut structure of the dome matched the scan data incredibly well. When it came to matching the animas, insects and mammals etc almost everything. Needed a nudge into place, not by much most position and rotations were off a little and some cutouts were swapped around completely. It took 2 days to painstakingly check and tweak the position XYZ and rotation XYZ do each of the 243 species until each position matched the scan data.

Moving the position and rotation for each species

Once all the 3d to scan data was done i generated different color lineup grids for each species and a wireframe from the 3d mesh to help with the projection lineup. CT were providing the projectors for the event and Nico was in charge of servers and lineup. With the scan and the modeling i was expecting that it should be straight forward, but with all these things you never really know until you you get all the hardware software and structure together. On the first projection alignment pass it popped straight in, there was no reason for me to double that it was going to “click in”

I did an initial test with disguise and quick cal to make sure the whole process worked than I handed it over to Nico to finish and tweak the alignment to which he did a great job. All along side this Zakk from Luke halls studio was programming the show on a parallel copy of the project, so once we had a decent alignment we were able to preview bits of the show with rest of the team.

It was a great project to work on and great to be working with mates and colleagues who really care about the job. I never actually got the chance to see the finished show as i had to leave London onwards to the next job, on all accounts and from pictures it looked great :).

Posted in Uncategorized | Tagged , , , , , , , , , , | Leave a comment

Protected: The Cube 2020

Posted on April 6, 2022 by andy

This content is password protected. To view it please enter your password below:

Posted in Uncategorized | Enter your password to view comments.