1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Hey Guest, If you'd like to instant message with many other users, then join our
    Official Rune-Status Discord Server!
    Dismiss Notice
  3. Got your own Server? Looking for a Server to play right now??
    Make sure you check out the Public Server Directory!
    Dismiss Notice
  4. If you're new to the Rsps Scene and want to jump straight into learning how to Set-up and Customise your own RuneScape Private Sever then check out the Guides and Tutorials for RS2 here.
    Dismiss Notice
Dismiss Notice
Hey Guest,
Are you sure your posting in the Right Sections?
If your unsure which RuneTek version you are working with visit this page to find out.

rev.317 Packing OSRS Data

Discussion in 'Guides & Tutorials' started by Ian, Jan 12, 2017.

  1. Hey guys,
    In this guide I will go through and explain how to pack data from newer RuneTek 3 revisions (e.g. OSRS rev.129) into older RT3 revision Client (e.g. RS2 rev.317) cache stores. I will try to be as detailed on each step as I am capable. At the end of this guide you should be confident in your ability to add data from any OSRS revision cache stores into your RS2 rev.317 Clients.

    Throughout this guide we will be using a handful of different tools, some of which you will have used before some possibly not. However if you have or haven't I will explain how to use each tool as we come accross it.
    Tools to download:
    First thing we will do is download the latest Oldschool RuneScape Cache directly, and for this we will use the OSRS Cache Downloader which was made by @SapphirusBeryl (at the time of writing this post OSRS is on rev.129 which requires us to use version 1.0.3+ of the Cache downloading tool which was quickly updated by the original developer asoon as he noticed the break.)

    Now the download link in the resoruces section for the OSRSCD is a link to the owners Git Repo. You will need to download the latest version from there as of writing this post it is version 1.0.3, only the source for the release is available for this so you will need to create your own runnable jar to run the program or alternativly run through your IDE. I use Eclipse as my defauly Java IDE and I prefer to make a runnable jar than run though the IDE. To do this simple click File -> Export -> Java -> Runnable Jar and follow on-screen instructions to complete. Though this is not a nessesity.

    Once we are in a place where we can run the program - run it through the Windows Shell or though the IDE to have an idea of what is going on, there is no UI for this program so running a Jar version of the program as a standalone program it will run in the background.

    Note: To run a Jar executable through the shell we simply create a file as:
    java -jar FILENAME.jar
    - though I feel you should already know this.

    When the program has finished running, you'll know when it has stopped printing anything into the terminal window, you should now have a new folder, ./data in the programs folder containing the newest OSRS revision cache files:
    rsps
    - OSRS rev.129 cache
    Now we have a fresh, clean copy of the cache from Oldschool RuneScape we can now move forward to extract data from the cache. There are only a handful of tools arround which are capable of dumping data from the OSRS RuneTek 3 revisions which are readable by the older RS2 Clients. The one we will be using is OSRS Cache Tool by @Poesy700.

    After downloading the OSRS Cache Tool, which is just a runnable Jar file (OSRS Tool.jar) you can run this straight off the mark no messing about, it has a well formed UI which is extremely easy to understand.
    rsps
    - Poesy700's OSRS Cache Tool

    The first thing to do with this tool before it become of any use atall is to load the cache that we dumped before so it is able to do as the buttons are labeled. You can do this by clicking File -> Load Cache and navigating to the folder which contains the Cache files. There is no alert of when it has successfully loaded the cache so once done the only way to know is to dump some of the data.

    We will need to dump all the data from the cache that the Tool is able too so what I advise you to do is to make a new Folder where you are happy to dump the data, I made a folder on my desktop named /129_Dump and inside i created folders for each of the dumps to stop myself getting lost further on.
    rsps
    - Dump store directory

    Once you have a place where you are comfortable in storing the dumps where you are able to locate easily later on we are able to move on in the guide. Now as said before we need to dump data from the cache which we can use in the rev.317 cache.

    You will need to click each of the buttons on the OSRS Cache Tool and navigate the tool to the folder of which you would like it to dump the data:
    rsps
    - Selecting export directory for 'Dump ItemDefinitions'

    You need to do this for each and check the program has successfully dumped the data;
    • Dump ItemDefinitions
      • obj.dat, obj.idx
    • Dump NpcDefinitions
      • npc.dat, npc.idx
    • Dump ObjectDefinitions
      • loc.dat, loc.idx
    • Dump AnimationDefinitions
      • seq.dat
    • Dump GraphicsDefinitions
      • spotanim.dat
    • Dump VarbitDefinitions
      • varbit.dat
    • Dump FloorDefinitions
      • flo.dat
    Note: You may also want to dump the models and animations from the cache with Poesy700's OSRS Cache Tool incase you wish to view them in a model viewer, I did this to make sure the models were infact what I was looking for you can see that here:
    rsps
    - RSMV version 0.3 by Xaves (Model 32797: Sorry you must be logged in to read this.
    Click here to log in.
    )

    Now we should have most of the data we need from the OSRS Cache, there is one more major thing we need to dump from OSRS and that is the Maps, typically a cruicial part in customised servers as a selling point they have newer areas in their rev.317. To do this though there is a problem... we need map data directly from Oldschool RuneScape, this data comes in the form of Sorry you must be logged in to read this.
    Click here to log in.
    , and the only known way to retreive these is to use a customized OSRS Client which will dump them as you play the game, every time you walk into a new reigon on OSRS the XTEA is as the OSRS Server sends them.


    I won't go into much more detail on getting XTEAs, you can find these readily available online a short time after the newer OSRS Revision has been released. Though its worth noting that you only need to collect them as you go - You NEED all the XTEAs for each revision, these barely change per update so you can create a store for your own XTEAs and basically drop in the new ones as you go to keep a full XETA map so to say, but we will come back to adding maps later as its the most difficult part of this whole process. For now we will deal with the changes we need to make to our own rev.317 client.

    Sometimes for revision updates you will be required to change the 'readValues' for the newer data, ofcause at this point in time the revision is rev.129 turning rev.130 so if you haven't done so before hand you will need to do it now.

    So to do this you will need to make changes to your rev.317 ItemDef.java to enable the new data to load without and issues so to do this we need to find the readValues and you can find these by devling inside the latest OSRS Deob, which in this case is rev.129, so being a deob the files are non-renamed and this makes it a little bit harder for people to locate what they are looking for, now i find that opening the deob inside my IDE and searching for specific things that are barely ever changed always gets me to where I want to be, now im looking for ItemDef.java and it would be so much easier if the file was named ItemDef.java but as per usual, life gives u a kick to the balls and it's far from named that.

    Now I myself know what a typical ItemDef.java contains and I know it contains a String similar to this:
    String[]{null, null, "Take", null, null};
    Though there are probably better methods to finding the ItemDef.java file, So searching for this only leaves me with one result in a file named: Class70_Sub16_Sub21
    Now searching through this file im looking for the method which reads data from the cache stream per index, this is typically the first method you see, for the rev.129 deob im using its named: method873, and it looks like this:

    Sorry you must be logged in to read this.
    Click here to log in.
    - osrs itemdef example.

    The next thing we need to do now we know the readvalues are to conver them form an unnamed version to the renamed version which will fit your rev.317 client.
    var1 is a reference to the class Class70_Sub22 which I know is the Stream class in the renamed rev.317 so what you should now do is sort out which method of streaming data in the Deob relates to which method in the renamed rev.317 for example:
    method616() = readSignedByte()
    I think you will then find it alot easier to match the readValues from the Deob to the renamed Client readValues: as an example i finished the method from the OSRS rev.129 and sorted the RS2 rev.317 clients ItemDef.java readValues method:
    private void readValues(Stream stream) {
       do {
           int i = stream.readUnsignedByte();
           if (i == 0) return;
           if (i == 1) modelID = stream.readUnsignedWord();
           else if (i == 2) name = stream.readString();
           else if (i == 3) description = stream.readString();
           else if (i == 4) modelZoom = stream.readUnsignedWord();
           else if (i == 5) modelRotationY = stream.readUnsignedWord();
           else if (i == 6) modelRotationX = stream.readUnsignedWord();
           else if (i == 7) {
               modelOffset1 = stream.readUnsignedWord();
               if (modelOffset1 > 32767) modelOffset1 -= 0x10000;
           } else if (i == 8) {
               modelOffset2 = stream.readUnsignedWord();
               if (modelOffset2 > 32767) modelOffset2 -= 0x10000;
           } else if (i == 10) stream.readUnsignedWord();
           else if (i == 11) stackable = true;
           else if (i == 12) value = stream.readDWord();
           else if (i == 16) membersObject = true;
           else if (i == 23) {
               anInt165 = stream.readUnsignedWord();
               maleEquipOffset = stream.readSignedByte();
           } else if (i == 24) anInt188 = stream.readUnsignedWord();
           else if (i == 25) {
               anInt200 = stream.readUnsignedWord();
               femaleEquipOffset = stream.readSignedByte();
           } else if (i == 26) anInt164 = stream.readUnsignedWord();
           else if (i >= 30 && i < 35) {
               if (groundActions == null) groundActions = new String[5];
               groundActions[i - 30] = stream.readString();
           } else if (i >= 35 && i < 40) {
               if (itemActions == null)
                   itemActions = new String[5];
               itemActions[i - 35] = stream.readString();
           } else if (i == 40) {
               int j = stream.readUnsignedByte();
               originalModelColors = new int[j];
               modifiedModelColors = new int[j];
               for (int k = 0; k < j; k++) {
                   originalModelColors[k] = stream.readUnsignedWord();
                   modifiedModelColors[k] = stream.readUnsignedWord();
               }
           } else if (i == 78) anInt185 = stream.readUnsignedWord();
           else if (i == 79) anInt162 = stream.readUnsignedWord();
           else if (i == 90) anInt175 = stream.readUnsignedWord();
           else if (i == 91) anInt197 = stream.readUnsignedWord();
           else if (i == 92) anInt166 = stream.readUnsignedWord();
           else if (i == 93) anInt173 = stream.readUnsignedWord();
           else if (i == 95) anInt204 = stream.readUnsignedWord();
           else if (i == 97) certID = stream.readUnsignedWord();
           else if (i == 98) certTemplateID = stream.readUnsignedWord();
           else if (i >= 100 && i < 110) {
               if (stackIDs == null) {
                   stackIDs = new int[10];
                   stackAmounts = new int[10];
               }
               stackIDs[i - 100] = stream.readUnsignedWord();
               stackAmounts[i - 100] = stream.readUnsignedWord();
           } else if (i == 110) anInt167 = stream.readUnsignedWord();
           else if (i == 111) anInt192 = stream.readUnsignedWord();
           else if (i == 112) anInt191 = stream.readUnsignedWord();
           else if (i == 113) brightness = stream.readSignedByte();
           else if (i == 114) anInt184 = stream.readSignedByte() * 5;
           else if (i == 115) team = stream.readUnsignedByte();
       } while (true);
    }

    Another thing we need to do is fix the ItemDef.java where the client unpacks the archive, now for example using @Major's Renamed 317-client in ItemDefinition.java you will find a method named:
    init(Archive)
    all we need to do is simply change the readUShort() with which ever data streaming method is used in the OSRS client. Usually you can just replace the readUShort() with whichever you have replaced it with in the readValues method from before, in this case (rev.129) it will be replaced with readUnsignedWord().

    Example:

    public static void init(Archive archive) {
       data = new Buffer(archive.getEntry("obj.dat"));
       Buffer buffer = new Buffer(archive.getEntry("obj.idx"));
       count = buffer.readUnsignedWord() + 21;
       offsets = new int[count + 50000];
       int offset = 2;
       for (int i = 0; i < count - 21; i++) {
           offsets[i] = offset;
           offset += buffer.readUnsignedWord();
       }
       cache = new ItemDefinition[10];
    
       for (int i = 0; i < 10; i++) {
           cache[i] = new ItemDefinition();
       }
    }
    Note: the +50000 and +21/-21 extras are to enable the client to read higher values than the rev.317 is used too. Im not 100% sure if this is needed or know any details to why it works but i've noted them anyway. (If you have an understanding please share i will happily update)

    Another problem you will encounter is the fact that OSRS no longer uses a file named 'map_index' which you also need to repack to the rev.317 cache so as said by Stuart "we need to do this ourselves." to do this we need to make use of this bit of code which again thanks to Stuart was ripped from @Poesy700's tool, you can add this to a tool like OpenRS which should enable you to create a 'map_index' file:
    
            RandomAccessFile raf = new RandomAccessFile(exportPath + "/map_index", "rw");
            System.out.println("Generating map_index...");
            int total = 0;
            raf.seek(2);
            for (int squareX = 0; squareX < 256; ++squareX) {
                for (int squareY = 0; squareY < 256; ++squareY) {
                    int squareId = squareX << 8 | squareY;
    
                    int mapFile = cache.getFileId(5, "m" + squareX + "_" + squareY);
                    int landscapeFile = cache.getFileId(5, "l" + squareX + "_" + squareY);
                    if (mapFile == -1 || landscapeFile == -1) continue;
                    raf.writeShort(squareId);
                    raf.writeShort(mapFile);
                    raf.writeShort(landscapeFile);
                    ++total;
                }
            }
    
    - you will need to edit the exportPath.

    Though you may find this part to be easier when you can just download it from somebody else, for example Sorry you must be logged in to read this.
    Click here to log in.
    is the map_index index file for rev.129.


    Now that we have all the files we need the next thing to do is to import this data we have extracted and compiled from the OSRS Cache and game we need to import this data into our own Clients cache.
    To acheive this we use the rev.317 Cache Tool by @Poesy700.

    On opening the tool we need to import the cache from our rev.317, click File -> Load Cache and navigate to the cache store of your client (It's best practice to make a backup of the cache you have beore you start tinkering about with it.)

    Next we need to 'replace' the data inside the cache with the new data we have, so to do this we simply choose the file of which we want to replace and click the 'replace' button. (It's that easy.)
    rsps
    And we need to do this for each of the data files we have, Note that the majority of these are under the Config tab and the map_index can be found in version. Once you have replaced each of the files we are ready to load up our client and test the outcome.


    - Credit to @Demonz-Scape for the image, I'm too lazy to get my own.
    Now I know this is not the best of guides on the matter although it may acctually be the only one online, this guide may be missing some information and help should be saught otherwise. Hopefully though this guide will help you to acheive what we have set out too and it will remove the shroud of mystery on how its done somewhat. If you know of something that I have missed or you can correct me in anyway please do so and send me a private message, I would appreciate that.

    Thanks for reading.
    - Ian.
     
    #1 Ian, Jan 13, 2017
    Last edited: Jun 23, 2017
    Tags:
    • Informative Informative x 3
    • Winner Winner x 1
    • List
  2. This is exactly what I've been looking for. I went through the guide and pretty sure I did everything right, but when I load the client now I get "Connection error - Retrying in ..." Thoughts?
     
  3. Be sure to pack the map, model, music and animations. You can dump these directly from the cache and pack using any Cache editor. Note that maps for higher revisions are encrypted and need XTEAs for each to be readable in a 317 revision client. Good thing is that XTEAs never scramble so the same can br used for each revision unless that part of the map has been changed in the update. For example a map for Lumbridge in 474 wont work for OSRS because it had had the RT4 make over afterwards. Where map objects have been changed. Also be sure to update the readValues for each new archive you pack, Items, Objects, Animations, Graphics, Variables, Maps, ect. This is essential. If you struggle you can always use a client with preloaded osrs data for reference. The readValues only change once in a while.
     
  4. Something went wrong dumping items.
     
  5. aight im going to give it a go. gl me.
     
  6. why does my osrs cache tool not respond? lol..... as soon as i try to pack itemdefs
     
  7. I wouldnt know brother, try speaking to the developer?
     
  8. Fucking phenomenal tutorial, Ian!
    However, to make the readvalues easier you could use Nshusa's OSDC (OS Data Converter) for just the straight dumps of the osrs data readvalues that'll automatically convert it to rev.317. However, Nshusa's tools arent for beginners who don't know what they're doing yet. I gotta check your cache tool out also. Nshusa also has a really nice looking UI on his RSAM (RS Asset Manager). It's sleek and is easy to understand for intermediate people who've been messing with RSPS for quite a little bit of time.

    But, as i said. I'll check out your tools that you provided. Good shit though Ian
     
  9. I'll look into updating the thread then, thanks for the info.
     
  10. Thanks for this guide, will be using this for sure!
     
Loading...
Loading...