Meditations – April 8th 2019

In March 2018, I attended Train Jam and chatted there with Rami Ismail about a project he was starting up : contemplative, 5-minute games released day-by-day. I thought it was neat, but I didn’t commit to anything since my time for side-projects is super limited.

In late November 2018, I was asked by Jupiter Hadley whether I’d like to contribute to a super secret project she and others were working on… I asked about details, and realized it’s the same thing Rami pitched me months earlier : Meditations! Jupiter explained that I should spend no more than 6 hours on the project, and submit it before December 14th 2018. The timing worked out, so I decided to give it a go.

I picked April 9th as the date (it was moved back 1 day to April 8th after the fact), and started browsing my Google Photos to see what the heck happened on an April 9th in my life… 2013 jumped out as a memorable one. My then-girlfriend (now wife) MC and I went to Nara, Japan where we met bowing deer and strolled the city on a beautiful sunny day. The cherry trees were at the peak of their flowering time, and they shed pink petals everywhere.

Yours truly, making friends with the locals

So I figured I’d do a little vignette game with a tree, falling petals, and heavy emphasis on the soundscape. I wanted it to feel vaguely like Proteus, with a Boards of Canada kind of musical vibe. Other stylistic references I had in mind : Devine Lu Linvega‘s Kanikule, and @ktch0‘s Places.

I asked my friend and colleague Rodrigo Rubilar whether he’d like to do sound design and music. I originally planned to do it myself with my limited audio gear, but I was excited to work with him on a small project like this, and he’d produce quality results 10 times faster than I could! We jammed an afternoon on an ambient track with a few layers, and one-shot sounds for the leaves hitting the ground.

https://twitter.com/renaudbedard/status/1073678256304316416
Rodrigo’s home studio is a goldmine of analog audio gear

At the same time I started hacking at a fractal tree generator in Unity. It’s actually the first time I’d tried to write one, so it was interesting to see how the simplest algorithm could apply to 3D, with some randomness injected into it. I exposed a ton of parameters, and played with them until I was happy with the result.

Fractal tree generation in action

Then I started thinking about the petals, and how I wanted them to move in the air. I started with a basic Unity particle system and wind zones, but I suck at authoring those and I could never get the look I wanted. So I resorted to manually updating the petals with simple physics. But then I wanted a lot of petals… thousands of them. So I started toying around with Unity 2018’s job system.

Optimization ended up being the most interesting engineering aspect of the project for me, and the biggest time-sink. There is no way I respected the development time limit of 6 hours (probably spent more than 40 hours on it), but it was hard to let go. So I’d like to at least document what I ended up doing and why.

If you just want to look at the code, it’s all here.

Jobified update

I ended up with a single job that does most petal-related work :

  • Update physics based on wind parameters
  • Detect ground collisions
  • Detect player interactions (you can rise petals from off the ground by walking near them)

I want to update all the active petals every frame, so I schedule the job in the earliest Update callback (you can tweak that using Script Execution Order), and complete it in LateUpdate.

I wanted petals hitting the ground to trigger sounds, so the naive way of making this happen in Unity (and my initial approach) is to have one GameObject per petal, with an AudioSource component. The AudioSources don’t have to be enabled all the time, only when they’re playing a sound, but that does mean that I have to update their Transform so that the sound plays at the right 3D position.

It’s possible to access and update Transforms from within a job by implementing IJobParallelForTransform, and pass the transforms to the job using a TransformAccessArray. A major gotcha with the current version of the job system is that only root Transforms will be assigned to different threads. I originally had petals as child of an empty game object just for grouping, but then my job was using a single worker thread. Reading this forum post explained it, but it’s still baffling to me, and makes for a pretty hideous Hierarchy tab.

I ended up not using Transforms for petals at all, but more on this later.

One thing that I found interesting is how one returns data from the job to the game code. For instance, I want to know which petals should trigger a sound by returning a list of petal indices that touched the ground. You can declare a field in the job as NativeQueue<int>.Concurrent (which is part of the Collections package) , and enqueue data as the job runs on multiple threads. From the calling site, you need a NativeQueue<int> to be created, but when it’s passed to the job instance, you need to use the .ToConcurrent() method to make it usable. Then, once the job is complete, you can use .TryDequeue() to extract elements from it. Works great!

I enabled Burst on my job very naively, and spent zero time optimizing the generated code. I did take care of the basics : make sure to mark job fields appropriately as [ReadOnly] and [WriteOnly], use the SIMD-capable Unity.Mathematics package whenever possible. In the end, the job runs way faster than I need, so I just moved on.

Burst had a dramatic effect on the job’s performance. Here’s a profile of the same code running on the same system, with Burst compilation toggled off and on. The biggest noticeable difference is that LateUpdate does not need to wait on jobs at all when they are Burst-enabled, which means it runs fully in parallel!

Longest job without Burst enabled : 1.70ms
Longest job with Burst enabled : 0.05ms

Rendering

Note : even though I’m using Unity 2018.3, I did not use the Scriptable Render Pipeline in this project, so the following probably only applies for the legacy rendering pipeline.

I initially had every petal GameObject with a MeshRenderer, a simple quad mesh, and the Unity Standard Shader with instancing turned on. It definitely worked, but Unity spent a lot of time in culling and batching, and I thought I could do better… by short-circuiting it completely.

It might sound like rendering everything all the time is a bad call, but in this game, the worst-case scenario of all petals being visible at once is the very first thing you see. Might as well make it a fast general case!

One way to draw objects manually is to use Command Buffers. By specifying a mesh, a material and a bunch of transformation matrices, you can use the DrawMeshInstanced() method to draw up to 1022 (that number I got empirically, but it appears to be undocumented) instances at once, in a single call. There is no renderer on the game objects themselves, so Unity acts as if they don’t exist, apart from issuing draw commands.

My petal update job’s final step is then to bake the petal’s current position, rotation and scale into a 4×4 transformation matrix, so that I can use them in my command buffers later. But there it gets annoying :

  • DrawMeshInstanced() takes a Matrix4x4[], nothing else, so we’ll have to copy our NativeArray to a managed array.
  • We have to respect the batch size of 1022, so we have to use NativeSlice to take 1022-element slices of our NativeArray.
  • NativeSlice.CopyTo() copies element by element, and it’s very slow.

I found a faster CopyTo implementation on the Unity forums which I slightly modified, and it does perform much better, but it’s a lot of moving data around for no reason. I wish that NativeSlices were directly usable with command buffers.

I end up with two very similar command buffers : one for depth rendering inside shadow-maps and for the camera’s depth pass, and one for regular opaque rendering.
The depth rendering command buffer uses the ShadowCaster pass from the Standard shader, and hooks to the AfterShadowMapPass event on lights as well as the AfterDepthTexture event on the camera. I needed the camera depth pass for SSAO using Keijiro Takahashi’s MiniEngineAO Unity implementation.

Triggering audio

As I hinted to earlier, I ended up ditching GameObjects for petals entirely. The only reason I originally kept them around (and updated at all times) was for audio, but it’s a lot of data shuffling just for sporadic sound triggers, so I opted for a simple audio trigger object pool instead.

I pre-initialize 1500 (again, empirical) dummy GameObjects with a disabled AudioSource component that lay dormant in the pool until an audio event happens. At that point, I take one from the pool, enable audio, position it at where the impact happens, and play the audio clip. There is no update cost to having them around otherwise, and they are disabled & returned to the pool when the sound playback is complete.

Petals are then strictly represented by a data structure that lives in a NativeArray, not by a GameObject. And the update job becomes a IJobParallelFor, mutating elements of that array. Neat!

You may wonder how dramatic a performance difference this makes. So here’s a comparison, taken from the same Macbook, of the CPU time spent only scheduling jobs between the version using Transforms and the version without. Something about how Unity handles transform access to a job definitely incurs sizeable overhead.

Using Transforms (1.79ms spent scheduling jobs)
Without Transforms (0.03ms spent scheduling jobs)

Bending the tree

When I had the petals mostly working well, it still felt strange that the tree didn’t animate at all. I wanted branches to sway in the wind, and I figured I could make that happen easily in its vertex shader.
It turned out to be a lot trickier than I expected, for a few reasons :

  • The tree is generated as a single baked mesh, there is (ironically) no tree structure to it. No great reason for this apart from lack of foresight.
  • The petals aren’t really attached to branches, their position is just set to the endpoint of branches when generating the tree.

To make the smaller branches bend but not the trunk, I set a bendability parameter during generation (how far away from the trunk that vertex is) in the texture coordinates.

A visualization of the bendability parameter for a generation

I then define the a bend axis perpendicular to wind direction, a bend angle dependent on wind force, and a shake factor which is just a bunch of low-frequency and high-frequency sine functions multiplied together, to make it feel a bit more noisy & organic.
These variables are mapped to shader globals, and I used keijiro’s angle-axis to float3x3 HLSL function to apply this in the vertex shader of my tree. So far so good!

Where it got tricky is for petals. I need to differentiate when they’re attached or detached, so I can apply the wind bending selectively. I ended up using a MaterialPropertyBlock to have per-instance data (a single float) encoding this state.

In the vertex shader (which is part of a simple Surface Shader), it took me a minute to understand how to do world-space transformations on vertex data. The simplest way I found was to transform the vertex to world-space, and take it back in object-space when I’m done. Not super efficient… ¯\_(ツ)_/¯

The full vertex shader for petals, color-coded by the lovely Rider

On the application-side, I did use a NativeArray<float> for instance data, but I’m not sure it’s the easiest way to go about it. I don’t actually read from or write to it from within my update job, it’s all happening in the regular Update(). It allowed me, however, to re-use Slice() and CopyToFast() to make 1022-sized chunks for rendering.
One thing to note : unlike with the draw arrays, MaterialPropertyBlocks cannot be reused for multiple batches, so I create all the ones I need up-front and iterate through them.

Another thing I realized, is that I need to apply the exact same function I used in the vertex shader to my petals the moment they are detached from the tree. This effectively bakes the tree’s bend at that moment in time into their transform, and avoids them visually warping around as they are detached.

In the end it still looks kinda stiff, but it’s better than the cement tree I started with.

Closing thoughts

I’m super happy with the final result. It’s relaxing, soft, yet sometimes surprisingly intense. It ended up feeling a lot more mournful than I expected, but I’m fine with that. I really enjoy the visual transition from a mess of pink sheets of paper to a menacing, pointy web.
But since everything was done in a rush, there’s some things I wish I’d done differently.

In Feburary 2019 I stumbled upon this rather excellent tree by Joe Russ, developer on Jenny LeClue :

https://twitter.com/Mografi_Joe/status/1100507496572223489

It made me realize that my tree shape and animation could be so much better. Having the branch splits be less even, more jagged and well, tree-like, would add a lot. And by having the branches hierarchically laid out, I could have bent the sub-trees and produce a much more convincing wind swaying effect.

Spending so much time on performance optimization made me wonder if doing a whole forest of trees, instead of a single one, would be possible at all. Wouldn’t that be neat? But it brought additional headaches that I didn’t want to deal with.

But there’s something to be said about making a small thing, with a tiny scope, a reasonable amount of polish, and putting it out there. Just the fact that I was excited enough about making this that I convinced Rodrigo to tag along made it worthwhile.
So big thanks to Rami, Jupiter and everyone involved in the Meditations project for making this happen. And thanks as always to my wife MC for humoring me while I spent precious evenings & weekends time endlessly debugging my stupid tree. ♡

Addendum : So what about ECS?

You might be wondering why I went straight with the Job System but didn’t use Entities, since they were showcased in the Megacity demo and Unity is really selling it as The Way to do massive amounts of objects with acceptable performance.
I don’t really have a good answer to this, except that I was short on time and was familiar with the idea of data parallelism, but not so much with ECS… and didn’t want to spend time learning how to use ECS properly. And I’m stubborn and will go down a path I’ve set for myself even if it’s a bad idea.

There’s good chances that using ECS is the best way to do what I was trying to achieve, but I didn’t go down that path yet. It would be interesting to revisit the project and do it that way; maybe I will!
I’m also curious to hear from people who have tried Unity’s ECS. Would it really make this easier to design? Would the rendering portion be simplified? Let me know!

Installing MS-DOS 6.22 on a 486 without a floppy drive using a CF-to-IDE adapter

A guide for a very specific problem, but hey, I had trouble doing it and pieced out the solution from a bunch of sources and advice, so here’s a distillation of that.

I’m currently building a 486 “retro” PC for fun, out of parts sourced from a combination of my dad having it lying around in the basement, eBay, and generous friends.
My endgoal is to reproduce the PC setup I had as a kid; MS-DOS 6.22, QuickBasic 4.5, a bunch of DOS games and ANSI art demos.

This week I was at point where I had everything to get started : a motherboard and CPU, graphics card, IDE hard drive, power supply, and a Compact Flash to IDE adapter with a 2Gb CF card.
I did not, however, have a floppy disk drive nor a CD-ROM drive. And it turns out installing MS-DOS without a FDD can be a tricky thing!

The options I saw just Googling variations of “install ms-dos to usb” (since on the modern PC, the CF card is connected via USB) were as follows :

  • Use Rufus to make a boot USB device : that only works if you want FreeDOS (and it does work great for that), because Rufus is unable to make a MS-DOS 6 boot disk, as acknowledged by its author.
  • Use UNetBootin or dd or some other image burner to transfer a bootable disk image from AllBootDisks to USB : this probably does work, but not with CF-to-IDE adapters in my experience. Possibly because the disk/MBR layout of a fixed drive and a floppy must differ…? In any case, I was never able to make a bootable partition that way.

At this point I feel like I need to stress : this is not a sensible project. I could install FreeDOS or even Windows 98’s MS-DOS 7.0 relatively easily, but I don’t want to. There is no good reason.

Regardless, here’s the solution I ended up with. The whole thing assumes that the “modern PC” you’re setting all of that from is running Windows.
Here’s the gist : use a virtual machine to mount the CF card as a virtual hard drive, then use the real MS-DOS 6.22 installer diskettes to create a partition and install DOS on it.

In detailed steps :

  1. Grab the install disks for MS-DOS 6.22 (The Legacy PC Project hosts it currently)
  2. Install VirtualBox for Windows hosts
  3. Plug in your Compact Flash card (I have a 2Gb card, I’ve been told bigger than 2Gb can be problematic)
  4. Open a Command Prompt with Administrator rights (required for unmounting)
  5. Type the following to get the list of physical disks in your system.
    This allows you to know what the DeviceID of your Compact Flash card is (should look like \\.\PHYSICALDRIVE1)

    wmic diskdrive list brief
  6. Find the drive letter associated with your CF card (through the Explorer), and unmount it by typing the following. (replace DRIVELETTER by your drive letter)
    This allows VirtualBox to have raw access to the drive.

    mountvol DRIVELETTER: /p
  7. Change directory to the folder where VirtualBox is installed (most likely C:\Program Files\Oracle\VirtualBox)
  8. Type the following to create a raw virtual machine disk (.vmdk) that maps to the whole device of the CF card.
    (replace the N in \\.\PhysicalDriveN with the DeviceID you found earlier)

    vboxmanage internalcommands createrawvmdk -filename CompactFlashCard.vmdk -rawdisk \\.\PhysicalDriveN
  9. Start VirtualBox with Administrator rights (required for raw disk access)
  10. Create a DOS Virtual Machine (it’s under the “Other” type, Version is “DOS”), with default settings, except that when prompted about the hard disk, choose the “Use an existing virtual disk file” option and provide the CompactFlashCard.vmdk file created earlier
  11. Go in Settings for this VM, “Storage” section, click on the “Empty” first floppy drive and click on the diskette icon, and select “Choose Virtual Floppy Disk File…”. Then, browse to the first disk of the MS-DOS 6.22 install set.
  12. Start the VM, boot from floppy, and go through the installer.
  13. When prompted to switch to the next disk, you can choose it from the “Devices”, “Floppy Drive”, “Choose disk image…” menu item.

There should be nothing special in the MS-DOS setup process itself at this point, just make sure you allocated a DOS partition in the hard drive you’ve mounted and set it as active… and you’re done!
You’ll have to use Disk Management to eject the device for your CF card (which is safer than just yanking it), but you don’t really need to assign a letter unless you want to browse its files directly from Windows.

FEZ 1.12

A bit more than a year ago, I sent this email to Ethan “flibitijibibo” Lee :

fna

And work on FEZ 1.12 officially started.

The goal of this large update to the Windows PC/Mac/Linux version of FEZ was the following :

  • Cut dependencies to OpenTK, the platform framework used by FEZ on Windows. I have had problems with it from the start, from sound card detection issues to windowing problems, to VSync and fullscreen issues… I wanted to give SDL 2.0 a shot, to see if it fares better.
  • Have more efficient music streaming. PC + Mac versions of FEZ used a C# Ogg Vorbis decoder called NVorbis, which seemed like a good idea because it would run on all platforms. I also wrote the streaming code that uses NVorbis and OpenAL, and it made its way into the main MonoGame repository! But it’s also very slow, resource-intensive and heavy on disk access. So I wanted to look into a better solution that wouldn’t break music playback in areas like puzzle rooms and the industrial world.
  • Have a single codebase for all PC + Mac versions of FEZ. As it stood with 1.11, there was a slightly modified codebase for Mac and Linux that ran on a weird hybrid of MonoGame and what would become FNA, called MG-SDL2. The PC version ran on my fork of MonoGame ~3.0 which I did not do a great job of keeping up to date with upstream changes, because when I did it usually broke in mysterious ways. This is not great for maintenance, and centralizing everything on a clean FNA back-end, with as little platform-specific code as I could, seemed like a good idea.
  • Make it the Last Update. Since I shipped FEZ 1.11 I had little intention of making additional fixes or features to the game because I simply don’t have the time with a kid and a fulltime job… and working on FEZ is getting old after 9 years. So I did want to address problems that people have with the game, but I don’t want to do it for the rest of my life. I had spent enough time away from the game that I was somewhat enthusiastic about coming back to it, especially if it’s at my pace, and that it’s my last time doing so.

So I didn’t announce anything, I didn’t announce a date, and I slowly chipped away at making this humongous update to FEZ. It’s been in beta-testing with an army of fans, speedrunners and friends since late January 2016 and over 120 bugs have been reported and fixed.

You can read the full 1.12 change log here (it’s also bundled with builds of the game), but I wanted to cover in more detail a few big changes that caused more headaches than I had anticipated.

Singlethreaded OpenGL

FEZ uses a loading thread so it doesn’t block the draw/update loop while levels are loaded. This loading process includes file loading, but the bulk of load times are spent interpreting loaded data : building models, uploading textures to GPU memory, creating shaders, calculating helper structures for collision, etc.

The original XNA version of the game did everything on the loading thread and DirectX 9 somehow made sure that everything would be just fine, even if calls to the graphics driver were made on a thread that wasn’t the main thread.

From version 1.0 to 1.07 of FEZ (PC/Mac/Linux), I used the background OpenGL context that MonoGame provides, so that I didn’t have to retool all my loading code, or slow everything down by synchronizing to the main thread on every GL call. This worked fine… mostly, depending on the driver, on some platforms. My development setup worked pretty good, but I had reports of awful load times on Mac, and on AMD graphics card; clearly this wasn’t good enough.

In FEZ 1.08, I got rid of the background context in favor of a call queue for GL operations that could wait until the next draw to be done (in order), and blocking GL calls for the ones that needed to be done instantly. This was minimally invasive and worked pretty well, but slowed down load times for setups that ran fine before. Also, what is considered a GL call or not is not known or defined by FEZ at this point; MonoGame internally does a check whether we’re on the main thread for every XNA call that involves GL, and if so, uses a closure to defer the call to the main thread (usually by blocking). This was good enough, but not great.

When switching to FNA, I wanted to take advantage of its “Disable Threading” mode which boosts performance and lowers GC strain because it doesn’t need to check whether you’re on the main thread on every graphics function; if you guarantee that you’ll never, ever use them on a secondary thread, it takes your word for it! This means that FEZ would need to do deferral/blocking itself. The loading code had to be retooled and verified throughout the game.

I ended up using a simple method very similar to what MonoGame did : the Draw Action Scheduler. I got rid of the “I need this now!” blocking calls (e.g. loading the sky texture and then instantly after, sampling its pixels for the fog color), and made sure that FEZ could load and process everything on its loading thread before the first draw could be executed, which unqueues and executes these draw actions. To keep the smoothness benefits of having a loading thread, I had to tweak granularity; sometimes it’s better to have a bunch of smaller actions that can be run while the loading screen renders, instead of having one big task that causes lost frames.

Here’s a fun one : I didn’t want to change FNA’s code, and I wanted a Texture2DReader that’s safe to call on a loading thread… so I wrote a FutureTexture2DReader that does file reading inline, but then lets you upload the texture to GPU in a second step :

var futureAtlas = input.ReadObject(FutureTexture2DReader.Instance);
DrawActionScheduler.Schedule(() => existingInstance.TextureAtlas = futureAtlas.Create());

I also realized that there are a lot of hidden GL operations here and there, that only happen in some circumstances, and that can blow the game up big-time if you’re not careful. There’s no safety net in FNA’s no-threading mode, so you have to be really confident that it’s 100% covered. I’m pretty confident, after a year. :)

Screen scaling modes

The saga of FEZ resolutions and black-bars is hard to justify. The excuses have varied from “it was meant to be run in 720p, so we only offer multiples of that” to “okay I guess we can do 1080p but it won’t look great” to “okay I guess we don’t actually have to pillarbox, but sometimes we’ll still letterbox”.

I think most people will be happy with the implementation we chose to go with in 1.12 :

  • No more black bars. The handful of situations where black bars were still required (mostly because I had been lazy and assumed a 16:9 aspect ratio) have been reworked. The only downside is that you might see the vertical ends of a level if you try hard enough, but it’s worth the presentation overhaul. One sensible exception : if you use a resolution that does not match your display adapter’s aspect ratio in fullscreen, the game auto-detects it and adds black bars so that the game does not appear distorted.
  • You choose the scaling mode. Not literally 1x/2x/3x because specific levels have control over that, but you choose whether you want the intended zoom level, prefer pixel-perfect scaling (which may cause a wider-than-intended zoom), or want to compromise with a supersampled view at the intended zoom level. The latter option is my favorite because it has no impact on pixel-perfect resolutions (like 720p and 1440p) but, for instance, will render with a 1440p backbuffer in 1080p in order to provide minimal pixel crawl/jitter… and provides anti-aliasing in first-person mode and whenever rotating objects are used. It’s a bit softer so people might prefer not to use it; but it’s an option!

Visual interpolation between fixed timesteps

Let me tell you the story of a .NET API misunderstanding that has deep consequences…

Let’s say you use the TimeSpan.FromSeconds() factory method to make a TimeSpan object that represents the duration of a 60hz frame. You’d use it like this :

var frameDuration = TimeStep.FromSeconds(1 / 60.0);

And I wouldn’t blame you for it. The method takes a double, there’s no indication that it would do any kind of rounding… but if you read the documentation :

millisecond

It’s not because TimeSpan’s precision stops at milliseconds, it store everything in ticks. It makes no sense. But here we are.

For the 5 years of its development, FEZ was designed with a 17ms timestep because of this issue. The physics were tweaked with a 17ms fixed timestep in mind, and yep, it skipped frames like it’s nobody business. Because it ran at approximately 58.8235294 frames per second, instead of the intended 60.

To help with this in FEZ 1.08 (PC/Mac/Linux), I decoupled the update and draw calls such that more than one draw can be done for one engine update. This eliminates tearing with a 60hz V-Sync, but once or twice every second, the same frame is presented twice in a row to the screen, which makes the game feel jittery. It was relatively minor, so I let it slide.

Fast-forward to 1.12, in which I decide to try and support my fancy new 120hz monitor properly. Drawing frames twice isn’t exactly great, it does make the game match the monitor’s synchronization but it doesn’t look any better than 60hz. Then Gyoo is play-testing the game and notices the original issue, that the game jitters even at 60hz… and it sinks in that the same root cause is making the game locked at an update framerate that doesn’t make any sense. I have to do something about it.

There’s two ways to go here, and both are painful : interpolate, or switch to a variable time-step. I already had tried the latter option when still developing the game for Xbox 360 and it’s very hard to pull off. The potential for hard-to-reproduce physics bugs is real, and it would mean retesting the whole game many times until we get it right. However, some parts of the game are easy to transition to a variable timestep in isolation since they don’t depend on the game’s physics, or have no impact on gameplay. So I went with a hybrid solution :

  • Gomez uses interpolation. This means that for every update, the next frame’s position is also computed, and when Gomez gets drawn, he gets interpolated to the right position between those two frames depending on timing.
  • The camera, sky, moving platforms, grabbed/held cubes and cube bits uses a variable time-step. These could relatively easily be transitioned to compute their movement/position per-draw instead of per-update, which was an instantaneous boost to fluidity, especially at framerates higher than 60hz.
  • Everything else is still at 17ms fixed timesteps. It turns out that it doesn’t matter for most entities to have fully smoothed movement, especially at that chunky world resolution, and with a fully smooth camera. So I stopped there.

This sounds like a fun time, but I’ve been working on regressions that these changes caused since I started the task back in April 2016. There were a lot of corner cases, places where the camera’s position was assumed to be the same in matching update and draw calls, jittering stars and parallaxed elements and… the list goes on. But the result is there : the game looks really, really good in 120hz right now.

Additional reading

Ethan wrote a whole series of posts during FEZ 1.12 development that covers other things that the patch addresses :

Special Thanks

First off, huge, HUGE thanks to Ethan for making FNA in the first place, motivating me to finish this patch, and working tirelessly on it since day one. Your support, help and dedication blow me away. If you appreciate the work he’s put on FEZ 1.12, consider supporting him on Patreon.

And then, the amazingly helpful testers that have been doing 1.12 testing on their free time to help the project, in no particular order (and I hope I didn’t forget anyone!) :

Thank you all so much, you made this possible. <3

As you can see, a lot of them are speedrunners. I’m very grateful for the passion of the FEZ speedrunning community, watching things like Vulajin run FEZ at AGDQ last year was amazing; I wanted to make the game solid for you guys!

Thanks for reading, and please enjoy FEZ 1.12!

Archival of formspring answers

From June 2012 to August 2013, I had a ask me anything kinda page on Formspring, a social network/website that hosted such pages (similar to ask.fm, which I believe came in later). I didn’t realize that it was regarded as a pretty crappy site, so I stuck with it for more than a year… and later in 2013, it was sold and re-branded as Spring.me, started getting flooded with spam, and finally died last week, in September 2015.

Thankfully, I did export the answers I had written in March 2013, since Formspring added a tool to do so when they announced that they would close or rebrand. It’s a shame that I lost everything between March and August, but here’s what I saved anyway :

Edit : Thanks to Reddit /r/fez user utter_bodge, I was able to recover the text of the few Q&A that I was missing! The answering dates are wrong though, so disregard that.

Collected Formspring Answers

Enjoy!

I Know This (Global Game Jam 2015)

“I Know This” is a game I made for the Global Game Jam 2015 along with Gavin McCarthy (art, design), Adam Axbey (sound effects) and Matthew Simmonds (4mat) (music); I did programming and design. The name we chose for the team was Two’s Complement.

As seen on Kill Screen (who posted the first article on the game, thanks a million!), Polygon, Popular Mechanics, Rock Paper Shotgun and many more! We’re flabbergasted, and very grateful for how popular our silly jam game has gotten.

Downloads

Windows x86 (version 1.1) : iknowthis_win_v1.1.zip
Mac Universal (version 1.1) : iknowthis_mac_v1.1.zip
Linux Universal (version 1.1) : iknowthis_linux_v1.1.tar.gz

Soundtrack

4mat released the OST he made for I Know This on his Bandcamp, pay what you want!

Update Notes

v1.1 (2015-02-20)

  • Linux : Build now works! The Unity 5.0 beta version it was originally built with had issues with Linux, now built with RC2
  • Mac/Linux : Builds should be executable out of the zip or tarball without the need for chmod
  • Mac/Linux : File icons and names now display correctly, though they are not sniffed from the machine, it’s a pre-built list coming from mine
  • Mac : Hacking now affects percentages (yeah… I should’ve tested this a bit more thoroughly)
  • Fixed “magenta rectangle” overlay on Shader Model 2.0 or lower GPUs
  • Delete key works as well as backspace key to clear red characters
  • Tweaked Clicky interactions (6 honeypots instead of 5, the wrong text was triggered when finishing your first hack, mentions more clearly that red text needs to be removed)
  • Admin scan bar flashes and turns red when you’re running out of hacking time
  • Hacking timer now starts at 35 seconds (originally 30), and will get shorter or longer depending if you fail or succeed hacks (cheap adaptive difficulty!)
  • Added license file with credits and acknowledgements (released under a Creative Commons Attribution-Non-Commercial-ShareAlike license)

Version 1.1 should clear all known bugs except for the seemingly rare “return does nothing” and the “redtext has HTML code in it” bugs which I can’t reproduce accurately at the moment. Let me know if there are new/other issues! And thanks for your patience.

Straight outta Isla Nublar

Remember that one scene in Jurassic Park? The one where Lex hacks the computer system in order to lock a door and protect everyone from the raptors, and exclaims…

unix-jurassic-park

That was basically the whole premise for our game.

When I saw the movie as a kid, that scene (and the file system UI that Lex “hacks”) always stuck with me as a quintessential faux-futuristic Hollywood representation of how computers work. I learned a bit later that this GUI was not made for the movie, but actually existed on SGI workstations and was ported to Linux as well, so it’s more legit than it looks! But in the end, it’s still a really great artifact of 90’s VR hopes and dreams, in which everything is better in 3D, even file browsers. (and Web browsers, too)

fsn1

The Game

It starts with the same basic premise as the scene in the movie : you have to find a file. To make it more interesting than your average hidden object game, you need to hack specific Search Nodes (purple files) which, upon successful hacking, will help you narrow down which potential Golden Folder contains what you’re looking for. Don’t pick the wrong one though, all the other ones are full of viruses and bad stuff!
Fun fact : the filenames you’ll see in the game are lifted from your hard drive, and 8.3ified for formatting and retro-chic reasons!

hacking

Hacking involves mashing your keyboard until code appears, and hitting the return key where the line endings are, just like in real life. The hacking minigame was heavily inspired by hackertyper.net, a fantastic way to feel like you’re real good at making up C code on the fly. However, we gamified it (oh, the horror) by not letting you go further than line endings, and adding a timer.

As you hack (or fail to hack) search nodes, sentinels will spawn and start looking for you. If they catch you, they warp you back at the root folder. Not a huge punishment, but enough to make you at least a little careful.

clicky

And then there’s Clicky, your favourite Office Assistant ripoff. He means well, but he sometimes gets in the way… and hides a dark secret. :o

Closing Thoughts

I don’t know that the game really qualifies as a jam game, because I worked for many evenings after the jam to smooth out the rough edges, make better Clicky interactions, fix the endings and other various bugs. The party version of the game was without music, I asked 4mat to produce something for us after the fact, and we were so so so lucky to have him contribute the lovely tunes you can hear in the game.

This was also my first experience with Unity 5, but I barely touched what it can accomplish. I’d say that the Audio engine is really nice, ducking was painless to implement… and the new UI stuff (even if it’s 4.6 and not 5) was a joy to use compared to the old GUI system.

And Gavin is the best! First time jamming with him, and it was a great match of design sensibilities, work-mindedness and just plain fun. <3

littleBits Jams

If you follow me on twitter this will be no surprise, but I am completely obsessed with the littleBits modular electronics system, especially the synth bits that came from their collaboration with Korg which are geared especially towards music-making.

I bought my first Synth Kit in April 2014 and fell in love with it. It took less than three weeks for me to order extensions (because let’s face it, the base kit is pretty limitative) and ended up building an obscenely huge collection of modules over the summer.

I even gave a talk in Mexico City about littleBits (among other things) during DevHR, a local technology and games conference. (you can read the slides here)

I’ve been doing a lot of “jams”, basically just toying with modules until I got something that was good enough to show.
I learned a lot about how synths work, circuit design and signal processing by doing them, and it’s great to be able to play with something that does not involve a computer once in a while.

Initially I just posted Instagram or Vine videos of them, but recently I’ve bought myself an USB sound card and am posting them to SoundCloud so they got a bit more long-form.

The constraints for what I consider to be a “jam” :

  • Don’t work on it for more than 2 or 3 hours (usually closer to an hour)
  • Record everything live, no post-processing other than trimming the ends, and fading them in or out

And the beauty of an analog system is that I’d have a really hard time reproducing an earlier jam, so once it’s unhooked and put aside, it’s gone. All I have from it is the recording. I tend to try something specific when starting a new jam, too, so there’s no point to repeat myself.

Here’s a collection of the longer (1 to 3 minutes) jams I’ve recorded. I’ll update as I post more.

I plan to gather the best of them when I got enough to call it an album, get it somewhat mastered and release it somewhere. Stay tuned, if you enjoy that type of thing!

P.S. If you’re interested in the earlier stuff, here’s a chronological list of every littleBits audio recording I’ve posted outside of SoundCloud (I = Instagram, V = Vine, YT = YouTube, hover for description) : V, I, V, I, I, V, I, I, V, I, I, I, V, V, I, I, V, V, YT, V, V, V, V, V, V, V

Malisse

Malisse Logo

Malisse is a game I made at TOJam “Party like it’s 19TOJam9” in 2014 with Devine Lu Linvega, Rekka B., Dom2D and technobeanie as Les Collégiennes, with sound effects by dualryan.

The game is playable in the web player at its itch.io home : http://renaudbedard.itch.io/malisse

It uses Unity and the whole source code and assets are hosted in a public repository on GitHub, and released under a Creative Commons Attribution license.

Malisse screenshot

The gameplay is a two-player cooperative physics sandbox and puzzle game. The objective for both players is to clear a path for Malisse as she walks on a sinuous path in the world through the looking glass…

A bunch of rabbits trail behind her, but they get scared easily! Everytime Malisse bumps into an object she cannot climb, a rabbit will run away, but you recover one rabbit for every level cleared. If all the rabbits are gone, Malisse ends up alone and … cries herself to death? That’s the end of that play session, anyway!
Otherwise, the levels are chosen randomly from a pool of 11 hastily-authored levels which vary in difficulty. If you get stuck early on your first attempt, definitely give it another shot since you might find more palatable challenges.

It was an interesting game jamming experience for me in many respects : first time with a 5-person team, first time implementing sprite animations in Unity (using 2D Toolkit), and first time writing a tool for use inside the Unity editor — a spline tool for drawing roads quickly. We also had interesting last-minute collision issues, since we wanted Malisse to be able to climb slopes but didn’t want to resort to rigid bodies to have that done automagically. Spherecasting to the rescue, and lots of debugging! ^_^

If you’re wondering, the music was made during the jam by Devine (aka Aliceffekt), based on “When Johnny Comes Marching Home”, because it just fits marching so damn well.

Other than the web player, you can download the OS X and Windows standalone builds.

Enjoy! Had lots of fun building it. Closing with an outstanding band shot of us by Myriame Pilgrim!

LES COLLÉGIENNES

Pyramidwarf

Pyramidwarf is a game I made in collaboration with Samuel Boucher (alias Monsieur Eurêka) with music by Stefan Boucher at the Global Game Jam 2014 in the TAG Lab of the Concordia University, in Montréal. The version you can download here is a tweaked, split-screen version of the “party build” you can find on the GGJ website.

Windows : pyramidwarf-final-1.01-windows.zip [12 Mb]
Mac : pyramidwarf-final-1.01-mac.zip [12 Mb]

Pyramidwarf's glorious title screen

This was my first experience with Samuel in a jam, he’s a kickass vector artist currently working with Ko-Op on GNOG, which you should totally check out.

(the game was demoed at the Montréal Joue Arcade 11, too!)

We worked with Unity, as is the usual for me in game jams, and the initial idea was to make a stacking game where you’d either race the other player with a really unstable stack of little guys, or throw parts of your pyramid to the other player’s to break it up. And of course, this being a game jam, we didn’t do half of the stuff we planned for, and ended up with a super janky physics-based stacking game that happens to be silly enough to be fun!

Pyramidwarf Gameplay

On my side, it was my first time really exploiting Unity physics in a game. I’d done some basic rigid-body stuff and used character controllers (Volkenessen was physics based as well), but never hinges or physics-based multi-body animation. One of the fun/interesting parts to do under pressure was the walk animation using torque and impulses : the leg pushes itself up, angles up, the body gets a magical shove and the legs readjust themselves to stay upright. It’s definitely not physically correct, but it looks like a bunch of cardboard puppets and that’s exactly what we were going for!

To build the pyramid, dwarves need to go up somehow, and the way we solved that is just… teleportation. These little guys have magic on their side, and they can teleport to the first free spot of the pyramid to keep stacking up. This caused rigidbody overlapping problems that I sorta resolved by just testing a whole lot if something’s there before teleporting, and denying the move otherwise.

The “final 1.01” build I posted here is not bug-free, but it’s shippable, so here goes. I might come back to it and fix rendering issues, and maybe implement dwarf-throwing, because it still sounds so great in my head.

Enjoy!

RAIN+BOW

RAIN+BOW is game I made with Devine Lu Linvega as Les Collégiennes during TOJam Sixy Times in 2011, and lay on the backburner for many years until it was resurrected for a showing at Gamercamp in 2013, this past weekend! We finally took the time to add polishing touches and add gamepad support, so I’m happy to release this version as the final public build!

Windows : raincrossbow_win_final.zip [16 MB]
Mac : raincrossbow_mac_final.zip [21 MB]

RAIN+BOW Screenshot

What am I looking at?

RAIN+BOW is a bullet hell, or a shmup where you need to make your way through a sea of bullets and enemies and hopefully kill some of them in the process. There is no score tracking, but the game tracks how far you were able to get and remembers your highest “score” for this playthrough. You have no health; a single bullet hitting your head will kill you, but as is usual in those kind of games, the hitbox is much smaller than the character.

OH MY GAH

There are three weapons at your disposal :

  • RAIN (-) : The “machine gun” continuous firing weapon, which does little damage but will grind away at oncoming enemies
  • CROSS (+) : The auto-guided missiles, which you can’t aim but will target nearby enemies and will clear up enemies behind you
  • BOW (|) : The laser weapon, which does take its time to charge but inflicts major damage, especially effective against blue “shield” enemies and columns of weaker enemies

Sometimes you’ll see an orange cat presenting you a power-up for one of your weapons; definitely kill that guy and grab the powerup to get a more effective firing rate and effectiveness, and if you reach level 3, a temporary ultra-powerful rainbow firing mode for that weapon.

You might think that spamming all three weapons at once is a good idea, but the firing rate is halved for every simultaneous weapon, which means you should prioritize the one you think is best suited for the situation, and fire off CROSS missiles every once in a while.

RAIN+BOW was made in Unity, with the models made in Cinema4D by Devine, and original music also created on the jam site by him back in 2011. It doesn’t use textures at all (all flat-coloured materials), and we even avoided transparency in favour of rapidly flashing entities on and off, to give an even more psychedelic and retro-arcadey feel to the game.
The levels are randomly generated based on predefined “events” that are presented in a random order, with some variation, in a very simplistic Spelunky-like way. Game difficulty varies wildly because the algorithms for level generation are quite crude, but it ends up being challenging and fun every time, which is what counts,… right?

Enjoy! And let me know if you get crazy high scores. :)

Ogg streaming using OpenTK and NVorbis

August 18th, 2015 Update

This article could be an interesting reference for people trying to understand how you can submit your own buffers to do streaming audio with OpenAL, but the actual tools I’m using (NVorbis, OpenTK) are outdated and I can’t recommend them anymore.

If you’re looking for a modern C# way of doing the same thing, look at how the Song class is implemented with Ogg Vorbis support in Ethan Lee’s FNA library, using Xiph Vorbisfile and the DynamicSoundEffect API, especially if you’re trying to do this in a MonoGame- or XNA-like environment. It’s much faster, the codebase is cut by half, and much less threading pitfalls!

Original article follows…


Updated September 7th 2012 : New OggStream class with better support for concurrent stream playback.

I was looking for a suitable replacement for the audio streaming and compression capabilities of XACT when porting an XNA project to MonoGame, and it doesn’t look like there’s a clear winner yet. MonoGame contributors suggested NAudio, but it looks like work needs to be done to make it portable, and the sample code is a mess. FMod EX or competing commercial solutions are an easy but costly choice. So I turned to OpenAL to see if it can be a free and usable solution for streaming compressed audio with some DSP capabilities.
T’was a bit challenging, but not impossible! :)

Decoding OGGs

Out of the box, OpenAL doesn’t support being fed MP3 or OGG sources. There are extensions for those, but according to one implementation, they’re deprecated. So you need to handle decoding yourself and feed the PCM bitstream to OpenAL.

It sure would be nice to have a purely managed implementation of libVorbis, but it doesn’t exist, so there’s a dozen homemade decoders floating around open source code hubs in various states of workability. I was pointed to NVorbis by TheGrandHero on the TIGSource forums, and I haven’t found a better alternative yet. CsVorbis is another, but it doesn’t support streaming, all the decoding is done up-front, which defeats the purpose. OggSharp is just a fork of CsVorbis with XNA helpers, so nope. TheGrandHero also mentioned trying out DragonOgg but having problems with it.

NVorbis worked like a charm for me, but it’s pretty early and doesn’t support some features like seeking around the stream, so looping or restarting playback requires creating a new whole new reader/decoder. I also took some time to optimize the memory usage in my fork of the project.
07/09/2012 Update : Andrew Ward, the author of NVorbis, resolved the memory allocation problems that the version I forked off had, so I pulled the new changes out instead.

Streaming

Once you have some decoded data, you have to make OpenAL stream it. This is sort of tricky but welldocumented.

(this image shamelessly stolen from Ben Britten’s blog entry linked above)

The basic idea is the following :

  • Generate one OpenAL source for your sound file, like XACT cues
  • Generate 2 or more OpenAL buffers
  • Fill at least one of those with the first samples of the sound and enqueue it/them to the source
  • Start playback of the source; it’ll play all the buffers associated with it, in order
  • In a background thread :
    • Query the source to know whether buffers have already been processed
    • If so, dequeue those buffers, refill them with fresh data and re-enqueue them

In practice, since it involves threads, it’s a bit more obtuse than the pseudo-code, but OpenAL makes it relatively painless. The trick is to read enough data and often enough to avoid buffer underruns.

Then, if you want to loop the sound, it’s not as easy as setting the source’s “Looping” parameter to true, because the buffers never contain the full sound file. Instead of no longer feeding the buffers when you hit the end of the Ogg stream, you just start back at the beginning and feed continuously, which has the nice side-effect of being 100% gapless.

Filters and effects

Finally, I wanted to have one fancy effect that XACT provided : low-pass filtering. This is used extensively in FEZ as a gameplay mechanic, so I could hardly live without it in MonoGame ports.

Thankfully, OpenAL Effect Extensions (EFX) provide cross-platform effects including filters, at least in theory. In reality, this depends on whether the driver implementation supports them, and even the Creative reference Windows implementation doesn’t on my system.

I was able to find a software implementation that does though, OpenAL Soft, and it’s cross-platform, so that bodes well.
To override the installed implementation, just supply the software DLL in the application’s directory and voilà. Had no problems with it up to now, performance or otherwise.

Plus, it comes with a console application that outputs which EFX and other extensions are supported in this implementation. This is handy to detect whether the right DLL’s been used, and helped me figured out that the Creative implementation didn’t support any filter. Here’s what it should say :

Sample class

The result of all of this is a OggStream class that is in my fork of NVorbis on GitHub, which you can find here :

Update : Version 2.0 comes with a sample console application which allows you to test and visualize how different streams get buffered and when buffer underruns occur in a nice concise format. I’m really quite happy about it, give it a shot! Here’s how it looks :

Legend of the symbols that this app blurts out :

  • (* means synchronous buffering (Prepare()) has started, and ) means it ended.
  • . means that one buffer has been refilled with fresh samples
  • | means that there are no more samples to consume from the sound file
  • ! means that playback stopped because of a buffer underrun and had to be restarted
  • { and } represent calls to Start() and Stop()
  • [ and ] represent calls to Pause() and Resume()
  • L, F or f and V or v in prefix means respectively that the stream is looping, fading the low-pass filter in/out or fading volume in/out

My code has only been tested on .NET on Windows, but I don’t see why it wouldn’t work in Mono either.
Like all the unlicensed content on this blog, it’s public domain, but attribution is appreciated.