Skip to content

Commit

Permalink
Add support for 0x8302 (very WIP)
Browse files Browse the repository at this point in the history
  • Loading branch information
@h3x4n1um
h3x4n1um committed Jun 7, 2019
1 parent 8268e64 commit db73b20
Show file tree
Hide file tree
Showing 5 changed files with 124 additions and 43 deletions.
60 changes: 50 additions & 10 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,14 @@ Bytecode | Type | Note
`0x45` | [RTON_t](#rton-number) | RTON number
`0x81` | [String](#string) |
`0x82` | [Utf-8 string](#utf-8-string) |
`0x8303` | [RTID](#rtid) | RTON ID
`0x83` | [RTID](#rtid) | RTON ID
`0x85` | [Object](#object) |
`0x86fd` | [Array](#array) |
`0x86` | [Array](#array) |
`0x90` | [Cached string](#cached-string) |
`0x91` | [Cached string recall](#cached-string) |
`0x92` | [Cached utf-8 string](#cached-utf-8-string) |
`0x93` | [Cached utf-8 string recall](#cached-utf-8-string) |
`0xfd` | [Begin of array](#array) |
`0xfe` | [End of array](#array) |
`0xff` | [End of object](#end-of-object) |

Expand Down Expand Up @@ -103,12 +104,18 @@ Bytecode | Type | Note
* `82 [L1] [L2] [string]` where `[L1]` is **unsigned RTON number** characters in utf-8 and `[L2]` is **unsigned RTON number** bytes of `[string]`.
## RTID
### `0x8303`
* `0x8303` begin the RTID (RTON ID???) of RTON (cross-reference???).
### `0x83`
```
83 03 [L1] [L2] [string] [L3] [L4] [string 2]
```
* Format **RTID(`[string 2]`@`[string]`)**
* It has 2 strings after the RTID: `RTID(2nd_string@1st_string)`.
* `0x83` begin the RTID (RTON ID???) of RTON (cross-reference???).
* After `0x8303` is 2 strings format: `[L1] [L2] [string]` where `[L1]` is **unsigned RTON number** characters in utf-8 and `[L2]` is **unsigned RTON number** bytes of `[string]`.
* It has 2 subsets (`0x2` and `0x3`)
#### `0x3` Subset
* After `0x8303` is 2 strings format: `[L1] [L2] [string]` same as `0x82`
* Example:
```
Expand All @@ -127,6 +134,35 @@ Bytecode | Type | Note
}
```
#### `0x2` Subset (may not be correct)
```
83 02 [L1] [L2] [string] [U2] [U1] [4 bytes ID]
```
* Format: **RTID(`[U1]`.`[U2]`.`[4 bytes ID]`@`[string]`)** (this is my assumption, it may not correct)
* `[L1] [L2] [string]` is same as `0x82`
* `[U2]` is the second number in uid
* `[U1]` is the first number in uid
* `[ID]` the third (hex) number in uid
* Example:
```
52 54 4F 4E 01 00 00 00
90 09 6D 5F 74 68 69 73 50 74 72 83 02 0C 0C 51 75 65 73 74 73 41 63 74 69 76 65 00 01 7D A7 7B 6D
FF
44 4F 4E 45
```
* JSON decode:
```json
{
"m_thisPtr": "RTID(1.0.6d7ba77d@QuestsActive)"
}
```
## Object
### `0x85`
* Create an object as value
Expand All @@ -151,8 +187,10 @@ Bytecode | Type | Note
```
## Array
### `0x86fd`, `0xfe`
* Array begin with `0x86fd xx` and end with `0xfe`, where `xx` is the number of elements in array.
### `0x86`, `0xfd` and `0xfe`
* `0x86` is declare an array
* Array begin with `0xfd xx` and end with `0xfe`, where `xx` is the number of elements in array.
* Example:
```
Expand All @@ -179,7 +217,7 @@ Bytecode | Type | Note
## Cached String
### `0x90` and `0x91`
* `90 xx [string]` create a `[string]` that has EXACTLY `xx` **unsigned RTON number** of bytes.
* `90 xx [string]`, the `xx [string]` is just like `0x81`
* By using `0x90`, the string will push in a stack then it can be recall by `91 xx`, `xx` is **unsigned RTON number**-th element in the stack (starting from 0). Let's call the stack is ASCII_CACHE.
Expand Down Expand Up @@ -207,7 +245,7 @@ Bytecode | Type | Note
### `0x92` and `0x93`
* Very much like the **Cached String**, `0x92` and `0x93` different is use the utf-8 encode
* `92 xx yy [string]` create a utf-8 `[string]` that has EXACTLY `xx` **unsigned RTON number** of utf-8 characters with `yy` **unsigned RTON number** bytes.
* `92 [L1] [L2] [string]`, the `[L1] [L2] [string]` same as `0x82`.
* Example:
```
Expand Down Expand Up @@ -248,6 +286,8 @@ Bytecode | Type | Note
```
## TODO:
* **Find the correct format of 0x8302**
* Support regex input e.g: `rton-json *.rton`
* Check for endianness
Expand Down
51 changes: 36 additions & 15 deletions rton-json/json2rton.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,16 @@ const uint8_t signed_int = 0x20; //int32_t
const uint8_t unsigned_int = 0x26; //uint32_t
const uint8_t float64 = 0x42;
const uint8_t null = 0x84;
const uint8_t rtid = 0x83;
const uint8_t object = 0x85;
const uint8_t arr = 0x86;
const uint8_t ascii = 0x90;
const uint8_t ascii_stack = 0x91;
const uint8_t utf8 = 0x92;
const uint8_t utf8_stack = 0x93;
const uint8_t eoa = 0xfe; //end of array
const uint8_t eoo = 0xff; //end of object

const uint16_t rtid = 0x0383; //rtid 83 03
const uint16_t arr = 0xfd86; //array 86 fd
const uint8_t arr_begin = 0xfd;
const uint8_t arr_end = 0xfe;
const uint8_t object_end = 0xff;

//import from main.cpp
std::string to_hex_string(std::vector <uint8_t> a);
Expand Down Expand Up @@ -86,22 +86,41 @@ int write_RTON_block(json js){
case json::value_t::string:{
std::string temp = js.get<std::string>();
//rtid
if (std::regex_match(temp, std::regex("(RTID()(.*)(@)(.*)())"))){
debug_js["RTON Stats"]["List of Bytecodes"].push_back(to_hex_string(output.tellp()) + ": " + to_hex_string(rtid % 0x100));
if (std::regex_match(temp, std::regex("RTID(.*@.*)"))){
output.write(reinterpret_cast<const char*> (&rtid), sizeof rtid);

//delete "RTID(" and ")"
temp.erase(temp.end() - 1);
temp.erase(0, 5);
//get 2 strings
std::string first_string = temp.substr(temp.find("@") + 1),
second_string = temp.substr(0, temp.find("@"));

uint8_t subset;
if (std::regex_match(second_string, std::regex("\\d+\\.\\d+\\.[0-9a-fA-F]+"))) subset = 0x2;
else subset = 0x3;

debug_js["RTON Stats"]["List of Bytecodes"].push_back(to_hex_string(output.tellp() - 1) + ": " + to_hex_string(rtid*0x100 + subset));
output.write(reinterpret_cast<const char*> (&subset), sizeof subset);
write_unsigned_RTON_num(int2unsigned_RTON_num(get_utf8_size(first_string)));
write_unsigned_RTON_num(int2unsigned_RTON_num(first_string.size()));
output << first_string;
write_unsigned_RTON_num(int2unsigned_RTON_num(get_utf8_size(second_string)));
write_unsigned_RTON_num(int2unsigned_RTON_num(second_string.size()));
output << second_string;
if (subset == 0x2){
uint64_t first_uid = strtoull(second_string.c_str(), NULL, 10);
second_string = second_string.substr(second_string.find(".") + 1);
uint64_t second_uid = strtoull(second_string.c_str(), NULL, 10);
second_string = second_string.substr(second_string.find(".") + 1);
int32_t third_uid = std::stoi(second_string, nullptr, 16);

write_unsigned_RTON_num(int2unsigned_RTON_num(second_uid));
write_unsigned_RTON_num(int2unsigned_RTON_num(first_uid));
output.write(reinterpret_cast<const char*> (&third_uid), sizeof third_uid);
}
else{
write_unsigned_RTON_num(int2unsigned_RTON_num(get_utf8_size(second_string)));
write_unsigned_RTON_num(int2unsigned_RTON_num(second_string.size()));
output << second_string;
}
}
//normal string
else{
Expand Down Expand Up @@ -177,12 +196,14 @@ int write_RTON_block(json js){
}
//array
case json::value_t::array:{
debug_js["RTON Stats"]["List of Bytecodes"].push_back(to_hex_string(output.tellp()) + ": " + to_hex_string(arr % 0x100));
debug_js["RTON Stats"]["List of Bytecodes"].push_back(to_hex_string(output.tellp()) + ": " + to_hex_string(arr));
output.write(reinterpret_cast<const char*> (&arr), sizeof arr);
debug_js["RTON Stats"]["List of Bytecodes"].push_back(to_hex_string(output.tellp()) + ": " + to_hex_string(arr_begin));
output.write(reinterpret_cast<const char*> (&arr_begin), sizeof arr_begin);
write_unsigned_RTON_num(int2unsigned_RTON_num(js.size()));
for (auto i : js) write_RTON_block(i);
debug_js["RTON Stats"]["List of Bytecodes"].push_back(to_hex_string(output.tellp()) + ": " + to_hex_string(eoa));
output.write(reinterpret_cast<const char*> (&eoa), sizeof eoa);
debug_js["RTON Stats"]["List of Bytecodes"].push_back(to_hex_string(output.tellp()) + ": " + to_hex_string(arr_end));
output.write(reinterpret_cast<const char*> (&arr_end), sizeof arr_end);
break;
}
//error
Expand All @@ -199,8 +220,8 @@ int write_RTON(json js){
write_RTON_block(i.first);
write_RTON_block(i.second);
}
debug_js["RTON Stats"]["List of Bytecodes"].push_back(to_hex_string(output.tellp()) + ": " + to_hex_string(eoo));
output.write(reinterpret_cast<const char*> (&eoo), 1);
debug_js["RTON Stats"]["List of Bytecodes"].push_back(to_hex_string(output.tellp()) + ": " + to_hex_string(object_end));
output.write(reinterpret_cast<const char*> (&object_end), 1);
return 0;
}

Expand Down
10 changes: 4 additions & 6 deletions rton-json/main.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ template<class K, class V, class dummy_compare, class A>
using workaround_fifo_map = nlohmann::fifo_map<K, V, nlohmann::fifo_map_compare<K>, A>;
using json = nlohmann::basic_json<workaround_fifo_map>;

const std::string ver = "2.4.0";
const std::string ver = "2.5.0";

json json_decode();
int rton_encode();
Expand Down Expand Up @@ -109,9 +109,7 @@ int main(const int argc, const char* argv[]){
}
debug_js["Info"]["File"] = file_path;

//remove extension
std::string file_name = file_path.substr(0, file_path.find_last_of("."));
debug.open(file_name + "_log.json");
debug.open(file_path + "_log.json");

//init RTON Stats
debug_js["RTON Stats"]["RTON Version"] = 1; //not sure if it ever higher than 1
Expand All @@ -125,7 +123,7 @@ int main(const int argc, const char* argv[]){
//read
input.open(file_path, std::ifstream::binary);
//write
output.open(file_name + ".json");
output.open(file_path + ".json");
output << std::setw(4) << json_decode();
//close
input.close();
Expand All @@ -137,7 +135,7 @@ int main(const int argc, const char* argv[]){
//read
input.open(file_path);
//write
output.open(file_name + ".rton", std::ofstream::binary);
output.open(file_path + ".rton", std::ofstream::binary);
rton_encode(); //write directly to file
//close
input.close();
Expand Down
2 changes: 2 additions & 0 deletions rton-json/rton-json.cbp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
</Compiler>
<Linker>
<Add option="-s" />
<Add option="-static-libstdc++" />
<Add option="-static-libgcc" />
<Add option="-static" />
</Linker>
</Target>
Expand Down
44 changes: 32 additions & 12 deletions rton-json/rton2json.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
#include <fstream>
#include <iomanip>
#include <iostream>
#include <sstream>

#include "fifo_map.hpp"
#include "json.hpp"
Expand Down Expand Up @@ -187,7 +188,7 @@ json read_RTON_block(){
res.push_back(num);
break;
}
/*I guess it is 0.0 in double base on previous the cheatsheet (type + 0 in that type)???
/*I guess it is 0.0 in double via previous bytecode in the cheatsheet (type + 0 in that type)???
but if 0x43 is 0.0 in double then wtf is 0x41???*/
//0.0???
case 0x43:{
Expand Down Expand Up @@ -228,10 +229,10 @@ json read_RTON_block(){
}
//RTID
case 0x83:{
uint8_t check;
input.read(reinterpret_cast <char*> (&check), sizeof check);
//check for 03
if (check == 0x3){
uint8_t subset;
input.read(reinterpret_cast <char*> (&subset), sizeof subset);
//subset 0x3
if (subset == 0x3){
//get 1st string
uint64_t s1_buffer = unsigned_RTON_num2int(read_RTON_num());
s1_buffer = unsigned_RTON_num2int(read_RTON_num());
Expand All @@ -244,8 +245,28 @@ json read_RTON_block(){
char s2[s2_buffer + 1];
input.read(s2, s2_buffer);
s2[s2_buffer] = 0;
res.push_back(std::string() + "RTID(" + s2 + '@' + s1 + ')');
res.push_back(std::string("RTID(") + s2 + '@' + s1 + ')');
}
//subset 0x2
else if (subset == 0x2){
uint64_t buffer = unsigned_RTON_num2int(read_RTON_num());
buffer = unsigned_RTON_num2int(read_RTON_num());
char s[buffer + 1];
input.read(s, buffer);
s[buffer] = 0;

uint64_t second_uid = unsigned_RTON_num2int(read_RTON_num());
uint64_t first_uid = unsigned_RTON_num2int(read_RTON_num());
uint32_t third_uid;
input.read(reinterpret_cast <char *> (&third_uid), sizeof third_uid);

std::stringstream ss;
std::string uid;
ss << std::dec << first_uid << '.' << second_uid << '.' << std::hex << third_uid;
ss >> uid;
res.push_back(std::string("RTID(") + uid + '@' + s + ')');
}
//unknown
else bytecode_error();
break;
}
Expand All @@ -261,10 +282,9 @@ json read_RTON_block(){
}
//array
case 0x86:{
//check for fd
uint8_t check;
input.read(reinterpret_cast <char*> (&check), sizeof check);
if (check == 0xfd){
uint8_t arr_begin;
input.read(reinterpret_cast <char*> (&arr_begin), sizeof arr_begin);
if (arr_begin == 0xfd){
size_t arr_size = unsigned_RTON_num2int(read_RTON_num());
json arr = json::array();
for (int i = 0; i < arr_size; ++i) arr.push_back(read_RTON_block()[0]);
Expand All @@ -286,7 +306,7 @@ json read_RTON_block(){
input.read(temp, buffer);
temp[buffer] = 0;
//logging
debug_js["RTON Stats"]["0x91 Stack"].push_back(to_hex_string(int2unsigned_RTON_num(stack_0x91.size())) + ": " + std::string(temp));
debug_js["RTON Stats"]["0x91 Stack"].push_back(to_hex_string(int2unsigned_RTON_num(stack_0x91.size())) + ": " + temp);
//push to stack_0x91 and write json
stack_0x91.push_back(temp);
res.push_back(stack_0x91[stack_0x91.size() - 1]);
Expand All @@ -307,7 +327,7 @@ json read_RTON_block(){
input.read(temp, buffer);
temp[buffer] = 0;
//logging
debug_js["RTON Stats"]["0x93 Stack"].push_back(to_hex_string(int2unsigned_RTON_num(stack_0x93.size())) + ": " + std::string(temp));
debug_js["RTON Stats"]["0x93 Stack"].push_back(to_hex_string(int2unsigned_RTON_num(stack_0x93.size())) + ": " + temp);
//push to stack_0x93 and write json
stack_0x93.push_back(temp);
res.push_back(stack_0x93[stack_0x93.size() - 1]);
Expand Down

0 comments on commit db73b20

Please sign in to comment.