with start/stop (or attach), existing threads do not use the proper syscall method

[derekbruening]

Running a forthcoming test api.detach, modeled after api.startstop, on 32-bit Linux, for #95:

Here's vsyscall:

interp: start_pc = 0xf779cbe0
  0xf779cbe0  51                   push   %ecx %esp -> %esp 0xfffffffc(%esp)[4byte]
  0xf779cbe1  52                   push   %edx %esp -> %esp 0xfffffffc(%esp)[4byte]
  0xf779cbe2  55                   push   %ebp %esp -> %esp 0xfffffffc(%esp)[4byte]
  0xf779cbe3  89 e5                mov    %esp -> %ebp
  0xf779cbe5  0f 34                sysenter  -> %esp
end_pc = 0xf779cbe7

Fragment 26, tag 0xf779cbe0, flags 0x1801030, shared, size 10, must end trace:
  0x55b95308  51                   push   %ecx %esp -> %esp 0xfffffffc(%esp)[4byte]
  0x55b95309  52                   push   %edx %esp -> %esp 0xfffffffc(%esp)[4byte]
  0x55b9530a  55                   push   %ebp %esp -> %esp 0xfffffffc(%esp)[4byte]
  0x55b9530b  89 e5                mov    %esp -> %ebp
  0x55b9530d  e9 27 fe 01 00       jmp    $0x55bb5139
  -------- exit stub 0: -------- <target: 0xf779cbe7> type: fall-through/speculated/IAT
  0x55bb5139  67 64 a3 00 00       addr16 mov    %eax -> %fs:0x00[4byte]
  0x55bb513e  b8 d8 90 b6 55       mov    $0x55b690d8 -> %eax
  0x55bb5143  e9 b8 8d fd ff       jmp    $0x55b8df00 <fcache_return> 

Thread 3 invoking futex and using OP_sysenter in thread-private do_syscall,so when it gets a signal
interrupting the syscall is says the PC is on the vsyscall page at the sysenter resumption point:

do_syscall:
  0x55cf6200  cd 80                int    $0x80
  0x55cf6202  a3 1c 5a c9 55       mov    %eax -> 0x55c95a1c[4byte]
  0x55cf6207  b8 6c fa 2a f7       mov    $0xf72afa6c -> %eax
  0x55cf620c  e9 ef fc ff ff       jmp    $0x55cf5f00 <fcache_return> 

Just updated syscall routine:
  0x55cf6200  0f 34                sysenter  -> %esp

cache_enter = 0x55b8de80, target = 0x55b95308
Exit from F26(0xf779cbe0).0x55b9530d (shared)   
 (block ends with syscall)
Entry into do_syscall to execute a non-ignorable system call
system call 240
fcache_enter = 0x55cf5e80, target = 0x55cf6200

master_signal_handler: sig=12, retaddr=0xf779cbd0
siginfo: sig = 12, pid = 30715, status = 0, errno = 0, si_code = -6
        xip=0xf779cbe9

Thread 4 is similar except the signal says it's at the post-OP_int in do_syscall:

do_syscall:
  0x55d3e200  cd 80                int    $0x80
  0x55d3e202  a3 9c 6e c9 55       mov    %eax -> 0x55c96e9c[4byte]
  0x55d3e207  b8 6c fa 2a f7       mov    $0xf72afa6c -> %eax
  0x55d3e20c  e9 ef fc ff ff       jmp    $0x55d3df00 <fcache_return> 

fcache_enter = 0x55b8de80, target = 0x55b95308
Exit from F26(0xf779cbe0).0x55b9530d (shared)   
 (block ends with syscall)
Entry into do_syscall to execute a non-ignorable system call
system call 240
fcache_enter = 0x55d3de80, target = 0x55d3e200

master_signal_handler: sig=12, retaddr=0xf779cbd0
siginfo: sig = 12, pid = 30715, status = 0, errno = 0, si_code = -6
        xip=0x55d3e202

The syscall method is exactly the reason -- the later threads all have
sysenter, but the earlier ones are not updated when the method is
"discovered", only the current thread is updated:

# grep -A 1 ^do_syscall: `ls -1td *0|head -1`/*
api.detach.0.30715.html:do_syscall:
api.detach.0.30715.html-  0x55b8ea00  cd 80                int    $0x80

log.0.30715.html:do_syscall:
log.0.30715.html-  0x55bfe200  cd 80                int    $0x80

log.10.30722.html:do_syscall:
log.10.30722.html-  0x55f0e200  0f 34                sysenter  -> %esp

log.1.30718.html:do_syscall:
log.1.30718.html-  0x55c46200  cd 80                int    $0x80

log.2.30716.html:do_syscall:
log.2.30716.html-  0x55c8e200  cd 80                int    $0x80

log.3.30719.html:do_syscall:
log.3.30719.html-  0x55cf6200  cd 80                int    $0x80

log.4.30723.html:do_syscall:
log.4.30723.html-  0x55d3e200  cd 80                int    $0x80

log.5.30721.html:do_syscall:
log.5.30721.html-  0x55d86200  0f 34                sysenter  -> %esp

log.6.30724.html:do_syscall:
log.6.30724.html-  0x55dce200  0f 34                sysenter  -> %esp

log.7.30720.html:do_syscall:
log.7.30720.html-  0x55e16200  0f 34                sysenter  -> %esp

log.8.30725.html:do_syscall:
log.8.30725.html-  0x55e5e200  0f 34                sysenter  -> %esp

log.9.30717.html:do_syscall:
log.9.30717.html-  0x55ea6200  0f 34                sysenter  -> %esp

This is unique to the start/stop interface, or to attach, b/c in regular
full control mode we'd see a syscall prior to a 2nd thread.

This causes several problems, such as state translation and safe spot
checking getting confused.

A related issue is that we're not translating when sysenter is the method
and a suspended thread is at do_syscall. Xref a84214e which should be
reverted and a proper translation solution put in place: we do need to
translate when at do_syscall when the method is sysenter, both when
do_syscall holds an int or a sysenter, for detach. This can happen w/o
this mixed-do_syscall if a thread just hasn't entered the kernel yet, but
is much more likely with the mixed.