> - how did you find what to optimise? - what approach did you use?

I used profilers: Chrome developer tools and Firebug. The Chrome profiler is very nice and easy to use, but it looks like it is a statistical profiler. When the usual invocation takes less than 1ms, it might get the times wrong (there might be a bottleneck that Chrome profiler is not showing).

The Firebug profiler is more precise, I believe it traces every executing instruction, which also makes it much slower to run you program in, but does sometime gives you interesting insights.

Also, there's intuition. I wanted to keep the code readable. The interpreter loop is basically something to the effect of

  while(true) {
    op = decode(mmu.read_32(cs.pc));
    cs.pc += 4;
    (optable[opcode])(op);
  }

which I believe is very readable, but it counters everything one expects about writing fast javascript code that runs in a tight loop. Bellard told me he thinks my "dynamic interpreted" version might be slower than a throughly optimized interpreter version, and he might very well be true. But I found the readability of the interpreter to be of the upmost importance, at least until I had a working "dynamic recompilation" version.

To optimize the interpreter version, the profiler showed me that basically three things take most (about 65, 70% of the time: * Instruction decoding * Memory access

There's not much I found to make instruction decoding faster , but the memory access time could be improved. In the beginning, I used the form mmu.read_32(addr) to read from any region of the memory, including RAM. Most of the accesses are of course from RAM, so it made sense to optimize for this fact (despite it being bad for readability).

So nowadays I do: op = (v8[rpc] << 24) | (v8[rpc + 1] << 16) | (v8[rpc + 2] << 8) | (v8[rpc + 3]);

For memory accesses.

Some other things I found randomly. The processor in the board I'm emulating has a 75Mhz clock. With that emulated frequency, Linux was spending a lot of time on the "idle task". Whan I told the emulator that the frequency was about 8Mhz, the problem went away.

Then, I thought to myself: it would be neat if I could implement parts of the libc in Javascript. That's how https://github.com/ubercomp/jslm32/blob/master/src/lm32_libc... came to be. So I implemented some string.h functions in javascript (memcpy, memmove, and memset), and voilá, the boot time went down about 3 times.

But in the end of the day, lm32_libc_dev is a hack with very negative consequences. I would have to modify every program I wanted to run on the emulated board to use my libc functions instead. I did this with the linux kernel and it was a bad experience. So I focused my energy on the dynamic code generation side of things.

I can tell you I have succeeded as the jslibc thing is now deleted on my local source three and I didn't experience any slowdowns whatsoever.

Thanks for this, it's a great piece of work and very easy to read. Nice code.

> "it looks like it is a statistical profiler. When the usual invocation takes less than 1ms, it might get the times wrong"

I don't understand this. As long as the statistical profiler can interrupt at any point, surely a routine running for 10k times for 0.1ms will still be interrupted a similar number of times as one routine running for 1s? Or are you thinking that there may be a bias to the sampling? (e.g. method of interrupting means we're not likely to see the first N ops after a method call or something)?

Did you see a significant difference in the profile between the two profiler approaches (firebug/chrome)?

> For memory accesses.

I wonder if your RAM accesses are mostly 32bit aligned and made in 4bytes at a time? (I guess you could instrument to find out)? If so, you might get a further speedup by having the RAM array stored as 32bit quantities instead of bytes, doing:

op = v832bit[rpc]; // only works for aligned code

and then having the RAM read/write 8/16 be special cases on top of that? That could also give a speedup on memmove etc (4x speedup?), making the libc hack less necessary?

> I don't understand this..

Well, I won't speculate on the workings of the Chrome profiler, but what I can say is that profiling in Chrome and Firefox gives different results (which may of course be caused by differences in the Javascript engines).

> I wonder if your RAM accesses are mostly 32bit

All accesses on the LM32 processor are aligned. Half word accesses have 2 byte alignment, and word accesses have 4 byte alignment. So in theory it would be easy.

I use ArrayBuffers for the RAM array (where supported). You can have different views for the same array buffer, so in theory I could have the ram ArrayBuffer with an 8bit view, a 16bit view, and a 32bit view, where each would be used for accesses of their size. In that sense, you're absolutely right and I think a good speedup would be achieved by that change (something like 2x speedup as most accesses are in fact 32 bit).

I don't do that for endianness reasons. The LM32 is big endian, and most devices nowadays are little endian. Unfortunately, the state of ArrayBuffer is still kind of chaotic, in that the endianness of the ArrayBuffer is the endianness of the host (they do this for performance reasons). So I can't do the v32[rpc] thing. Theoretically, I could if the programs have Read-Write consistency, i.e., if they only used the same size for reading and writing some data, which they don't. A program that tests the endianness of the processor, for instance, does not have read write consistency.

I could use ArrayBufferViews to access memory in an endianness-independent way, but I tried it and it's actually much slower.

To make memory faster, two things can be done either: 1) emulate a little endian processor instead. 2) have a fast byteswap operator in javascript, without a function call (never gonna happen).

So since 2 is never going to happen, the solution is to tackle 1.

Ugly thought - if "almost all" accesses are Read-Write consistent, maybe an exception list of addresses could be used?

Basically do the 4byte cell approach, and make most of your RAM the "fast" endianness, with some 4byte cells "correct" endianness, because you know they are being accessed in a way that cares. (Since you're interpreting, I'm guessing this is reasonably-easily detectable?)

It does put a "if-addr-on-exception-list" on your mem access path, but if that's a hash lookup it could be OK. If you can further guarantee that your text segments are always fast-endian, then your opcode dispatch loop can still go direct and miss out that test.

That would actually work, I guess :). I'm pretty sure the memcpy, memmove, etc. implementations I've seen for this architecture all work by copying one byte at a time (argh!) though, but that problem can be solved with some smart-if-ugly hackery.

But, as you said yourself, it would be kind of messy, in that it would be hard to be certain if it works in 100% of the cases.

I think it would be easier to just change the toolchain code and make it think the processor is little endian.