Walking Through Walls | Part One: Collision Detection in Shenmue

Today's topic was selected by the Phantom River Stone blog patrons via our monthly poll. If you'd like to have a say in what subjects we cover next, consider supporting us on Patreon!

The topic we'll be looking at today is about collision detection in Shenmue. The aim is to allow Ryo to "walk through walls" and roam around the landscape freely, without being constrained by the boundaries normally in place in the game.

The discussion will be divided into three parts:

• Part One (this post): some technical background of the knowledge contributed by Shenmue community members on the way collision data is stored in the Shenmue games, and a proposed approach for applying this to the Shenmue I & II re-releases.
• In Part Two, we will provide an easy-to-use file you can run to allow Ryo walk through walls for any area in Shenmue I (on PC). No technical expertise required!
• Part Three will introduce a tool for Shenmue II.

Note: although today's overview is somewhat technical, do not worry if you aren't able to follow everything; in the following posts of this series we will be providing a tool that takes care of all the necessary changes on behalf of the player.

Hacking & the Shenmue Community

The Shenmue fan community has a strong hacking / modding scene that goes back many years, right to the day the original games were released on the Dreamcast. This activity continues with vigor today, new graphic & sound mods appearing already for the recently-released PC (Steam) version of Shenmue I & II.

A way to disable collisions in Shenmue was not publicly announced until 2014, many years later than the original release of the games. It was first announced by community member Giorgio, who identified that most collision-related data is stored in map files, and by modifying this he was able to successfully remove collision detection in Shenmue 1. This inspired the search for a method that would work with Shenmue 2 (which, it turns out, has differences "under the hood") and shortly after one was discovered by BlueMue. Since then a number of fan-made exploration videos and mods have been created for the Dreamcast version, notably on BlueMue's YouTube channel and James Reiner's YouTube channel.

Collision Detection in Shenmue

The lure of disabling collisions (a term I think is more accurate than "no clipping") is that it lets the player move the normal boundaries to explore areas they're not normally supposed to see, or get closer to things that normally cannot be approached. In Shenmue, these boundaries prevent Ryo from walking through walls and buildings - or from running off the path in the forests of Guilin

This has come in useful in the past for articles on the blog, for example to get a better view of the fountain in the Hazuki Residence garden in Shenmue 1, or to observe the riverside village of Langhuishan from across the Li River in Shenmue 2.

Left: The Hazuki bamboo fountain in detail (Shenmue 1)
Right: Ryo in Langhuishan, seemingly unfazed by his newfound aquatic ambulation ability (Shenmue 2)

How Collision Data is Stored in Shenmue 1

Whenever Ryo enters a new area of the game (triggering the familiar "Loading..." screen), new data for the area's map and layout is loaded, and part of this data is information about the boundaries within which he may move in the game world.

First let's discuss the structure for layout for Shenmue 1 - as mentioned in the introduction, Shenmue 2 has some differences which we will address next.

There is a set of files that contain information about the layout of an area of the game all share the name MAPINFO.BIN, and there is a file in a separate subdirectory on the disc for each area. This file contains, among other things, the data that instructs the game engine where Ryo is allowed to move for that area. Our aim is to remove this data in some way or other.

This removal must be done precisely: if we remove too much data, we may end up taking away the information about ground heights, for example. Without this, it will make it difficult to properly explore the landscape as Ryo will simply maintain his starting level throughout the area and will walk "through" slopes rather than along them.

The subsection of the data we are interested in starts with a four-letter marker "COLI", which defines collisions in the horizontal plane. By removing the data within this subsection only, Ryo will still be able to interact with doors and steps, and follow the surface of the ground as he walks, while being able to walk freely through objects and walls.

Tom's Hot Dog truck from a fresh angle.

In order to remove the data in the COLI section, we need to know its length. This is conveniently stored in the 4 bytes immediately after the tag.

The illustration below shows a COLI tag, the four bytes specifying the length of the data (highlighted in green), and the raw data bytes that follow (highlighted in yellow).

Note: the length bytes are stored using a reverse-order system (known as "Little Endian") so we must mentally reverse their order to "Big Endian" to identify the data length. For example, if the four bytes are 00 00 54 D0 then the data length is 000054D0 hexadecimal bytes, or 0x54D0.

The COLI section data (in yellow) containing boundary information, within a MAPINFO.BIN file.

How Collision Data is Stored in Shenmue 2

Unlike Shenmue 1, where the collisions can be disabled very precisely leaving other environmental interactions in place, this is not the case with Shenmue 2 at the moment as its data structures differ from the first game and are not well understood currently.

However it has been found that for Shenmue 2, the main collision data is defined somewhere within a section of each MAPINFO.BIN file marked with "FLDD".

The best achievable outcome at this current time is to disable the FLDD section, which will give Ryo the ability to roam freely around the landscape of the current area. Unfortunately this section also contains information relating to things such as NPCs and triggering points for leaving the area.  So Ryo will not be able to climb stairs or walk on slopes (he will walk through them instead), or leave the area.

With no collisions, Ryo is able to sneak into Beverly Hills wharf through the closed gate...

New Findings

While experimenting in order to write this post, I discovered a few interesting findings that we will be able to incorporate into our solution:

• Data changes that worked for the Dreamcast versions of the game also work for the Shenmue I & II release, as the core game engine logic has been kept intact.
• For Shenmue I:
• Clearing the collision data by setting the bytes to zero is sufficient to remove collision detection. It has the same effect as deleting the section.
• Collisions for the second level of apartment buildings is stored in a separate COLI section. By clearing multiple COLI sections within the MAPINFO.BIN file we can achieve free roaming for all levels in the area..
• For Shenmue II:
• A possible way to disable collisions is to skip loading the FLDD section altogether, avoiding the need to modify individual MAPINFO.BIN data sections. Investigation is on-going into this.

Proposed Approach for Shenmue I & II

The traditional way, with the Dreamcast version of the games, has been to remove the identified collision data has been to make use of a hex editor to literally delete unwanted data bytes from each MAPINFO.BIN file, for which there is generally one for each area.

This may have proven too high a technical hurdle for some since making such edits is a large job, given the large number of MAPINFO.BIN files to be edited. In addition, there was the complication of having to create a new disc image that contained the new files.

The simpler alternative we will be taking is to apply the changes directly in memory using a memory hacking tool. This will be for the recent PC version of Shenmue I & II on Steam.

Next Time

While the discussion this time has been somewhat technical in nature, in our next post in Part Two, we will be introducing a tool for Shenmue I that takes care of all the under-the-hood changes needed: simply run it and have fun charging around the landscape, through walls and out of bounds.

To finish off, here are some sample images captured during testing of collision removal for Shenmue I & II.

Interestingly, there are textures of shrubbery in place around the outside of the house while exploring the interior. Since the paper windows of the house are slightly translucent, it gives the effect of the garden outside when viewed from inside.

Ryo wanders (literally) through his bedroom cupboard.

Ryo walks up to his house's kitchen door (normally a non-accessible area). 

A view of the road down from Sakuragaoka to Dobuita, normally hidden.

Looking across at the King's Road bridge from a normally-inaccessible area.

Ryo in the low-resolution version of a Hong Kong shop interior. These textures would normally only be glimpsed from a distance when the shop door opens..

Become a Patron!