make LISP=ccl fails

[jkordani]

osx 10.10 ccl64 build from trunk, make line was make DESTDIR="path/to/buildapp" LISP="path/to/dx86cl64" install

note hardcoded use of sbcl at frame 14.

(20E7C6A0) : 0 (>-2 NIL 0) 2293
(20E7C6D0) : 1 (<=-2 NIL 0) 29
(20E7C6F0) : 2 (FUNCALL #'#<#<CCL::STANDARD-KERNEL-METHOD PRINT-OBJECT (STANDARD-OBJECT T)>> #<SILENT-EXIT-ERROR #x302000D5F3FD> #<STRING-OUTPUT-STREAM #x302000D5EC5D>) 69
(20E7C710) : 3 (%CALL-NEXT-METHOD ((#) #<CCL::STANDARD-KERNEL-METHOD PRINT-OBJECT #> #<SILENT-EXIT-ERROR #x302000D5F3FD> #<STRING-OUTPUT-STREAM #x302000D5EC5D>)) 869
(20E7C790) : 4 (FUNCALL #'#<(:INTERNAL (PRINT-OBJECT (CONDITION T)))>) 221
(20E7C7B8) : 5 (%CALL-NEXT-METHOD ((#) #<CCL::STANDARD-KERNEL-METHOD PRINT-OBJECT #> #<SILENT-EXIT-ERROR #x302000D5F3FD> #<STRING-OUTPUT-STREAM #x302000D5EC5D>)) 869
(20E7C838) : 6 (FUNCALL #'#<(:INTERNAL (PRINT-OBJECT :AROUND (T T)))>) 221
(20E7C860) : 7 (%%CNM-WITH-ARGS-COMBINED-METHOD-DCODE #<SIMPLE-VECTOR 5> 69007642 NIL) 1029
(20E7C8E8) : 8 (FUNCALL #'#<#<CCL::STANDARD-KERNEL-METHOD CCL::REPORT-CONDITION (CONDITION T)>> #<SILENT-EXIT-ERROR #x302000D5F3FD> #<STRING-OUTPUT-STREAM #x302000D5EC5D>) 237
(20E7C928) : 9 (%BREAK-MESSAGE "Error" #<SILENT-EXIT-ERROR #x302000D5F3FD> 69007728 #>) 1101
(20E7CAF0) : 10 (BREAK-LOOP-HANDLE-ERROR #<SILENT-EXIT-ERROR #x302000D5F3FD> 69007728) 701
(20E7CB58) : 11 (%ERROR #<SILENT-EXIT-ERROR #x302000D5F3FD> NIL 69007728) 333
(20E7CB80) : 12 (FUNCALL #'#<(:INTERNAL MAIN)> #<BASIC-FILE-CHARACTER-OUTPUT-STREAM ("dumper-f0dIhgSY.lisp"/4 UTF-8) #x302000D76E5D> #P"dumper-f0dIhgSY.lisp") 597
(20E7CBB0) : 13 (CALL-WITH-TEMPORARY-OPEN-FILE "dumper.lisp" #<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL MAIN) #x302000D777AF>) 781
(20E7CC48) : 14 (MAIN ("sbcl" "--asdf-path" "/Users/joshuakordani/Source/buildapp/" "--load-system" "buildapp" ...)) 461
(20E7CC88) : 15 (CALL-CHECK-REGS BUILD-BUILDAPP) 221
(20E7CCC0) : 16 (CHEAP-EVAL (BUILD-BUILDAPP)) 101
(20E7CCF8) : 17 (FUNCALL #'#<(:INTERNAL CCL::EVAL-STRING CCL::STARTUP-CCL)> "(buildapp::build-buildapp)") 485
(20E7CD40) : 18 (STARTUP-CCL NIL) 1541
(20E7CDA8) : 19 (FUNCALL #'#<(:INTERNAL (CCL:TOPLEVEL-FUNCTION (CCL::LISP-DEVELOPMENT-SYSTEM T)))>) 61
(20E7CDC8) : 20 (FUNCALL #'#<(:INTERNAL CCL::MAKE-MCL-LISTENER-PROCESS)>) 661
(20E7CE60) : 21 (RUN-PROCESS-INITIAL-FORM #<TTY-LISTENER listener(1) [Active] #x3020004476CD> (#<CCL:COMPILED-LEXICAL-CLOSURE # #x3020004471FF>)) 669
(20E7CEF0) : 22 (FUNCALL #'#<(:INTERNAL (CCL::%PROCESS-PRESET-INTERNAL (CCL:PROCESS)))> #<TTY-LISTENER listener(1) [Active] #x3020004476CD> (#<CCL:COMPILED-LEXICAL-CLOSURE # #x3020004471FF>)) 573
(20E7CF98) : 23 (FUNCALL #'#<(:INTERNAL CCL::THREAD-MAKE-STARTUP-FUNCTION)>) 277

[xach]

Arg 0 in MAIN is ignored, so that's not the issue. Looks like an arithmetic error. I'll try to reproduce locally.

[jkordani]

I still have a lot to learn. I've been trying to figure out how to work in the slime debugger to work my way through backtraces like this but i seem to be having a hard time breaking through. If you do figure this out would you mind briefly describing how you came to figure out what the issue was?

[rprimus]

Wed Aug 5 00:24:29 BST 2015

@jkordani : Please perform the following (within a simple terminal/iTerm):

exec bash
script /tmp/buildapp.l0g
which ccl
cd /tmp
git clone [email protected]:xach/buildapp.git
cd buildapp
make LISP=ccl
ls -la buildapp
./buildapp
exit

If the make fails or there is no buildapp in the directory, please post/attach /tmp/buildapp.l0g to this issue.

It may also help knowing how ccl was installed on your system. Did you use Homebrew?

This should help @xach .

P.S.
I've found the following links very helpful and refer to them whenever I have issues:

http://www.catb.org/esr/faqs/smart-questions.html
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

[jkordani]

I get ccl from svn and track trunk, I usually keep stuff like that in a ~/Source directory, so ~/Source/ccl/. I have slime setup to go into this directly and load dx86cl64 usually, and I don't usually add ccl to my path, I just fire up emacs and go. On osx, I have to also open emacs from a terminal window in order for osx to provide emacs with the path environment variables, necessary for allowing things like lisp wrapper libraries to find binaries in macports.

output of buildapp.l0g can be found here:
http://paste.lisp.org/display/153025

[rprimus]

Wed Aug 5 01:28:32 BST 2015

Thanks for the ccl build info.

1. I have checked out stable:

% ../ccl-stable/dx86cl64
Welcome to Clozure Common Lisp Version 1.10-r16196  (DarwinX8664)!

as well as trunk:

 % ../ccl/dx86cl64
Welcome to Clozure Common Lisp Version 1.10-dev-r16089M-trunk  (DarwinX8664)!

and have been able to build buildapp successfully using both.
(output: http://paste.lisp.org/+3A2S)

Since we're both using 1.10-r16196 (DarwinX8664)!, I suspect the problem may lie elsewhere
(but I will defer to @xach as my lisp experience is < 1 week).

Additional info:
My OS details:

 % sw_vers
ProductName:        Mac OS X
ProductVersion: 10.10.4
BuildVersion:   14E46

linked libraries:

rprimus@rp-mbp /usr/local/src % otool -L ccl-stable/dx86cl64
ccl-stable/dx86cl64:
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)
rprimus@rp-mbp /usr/local/src % otool -L ccl/dx86cl64
ccl/dx86cl64:
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.11)

I'm also interested in the frame inspection and debugging.

[jkordani]

same sw_vers, same steps, including make clean on buildapp, error persists until I place ccl in my path. then building buildapp in lisp and from make both work. It make me wonder if the value of LISP=ccl is ignored, if I were to chose a different build of ccl for LISP, will the system only ever use the one found in path as ccl?

[rprimus]

Wed Aug 5 11:43:13 BST 2015

UPDATE:

My bad - I still had my original ccl in my $PATH.
I've also removed sbcl from path.

1. I found this issue: https://groups.google.com/forum/#!topic/quicklisp/q1sCAdevZ9s
2. Starting the REPL and simply loading the 'temporary' dumper (lisp) file created, buildapp is built successfully.

(I've been looking at each frame using:

:raw <frame-number>
:f <frame-number>

... continuing investigation

Wed Aug 5 14:39:04 BST 2015

I've narrowed it down to line 65 in dumper.lisp:

   (ccl
    :initarg :ccl
    :accessor ccl
    :initform "ccl")

which is referenced at line 401 in buildapp.lisp:

(let ((process (run-program #+sbcl (sbcl dumper)
                                            #+ccl  (ccl  dumper)

run-program is a macro defined in utils.lisp.

As it's just ccl (not an absolute path and not relative to current directory) failure.

[This also means that on OS X (with default homebrew install) buildapp executable/image will always be compiled with 32-bit version.]

(Am famished - will continue after lu-inner!)

[jkordani]

Just got there this morning as well. I'm using the slime debugger to jump around, and c-c c-w c-c to track who calls the different functions and methods to browse around. I think we need to make one issue tracking this instead, but I'm not looking to make life more difficult for the maintainer.

[rprimus]

On Wed, Aug 05, 2015 at 08:25:40AM -0700, Joshua Kordani wrote:

Just got there this morning as well. I'm using the slime debugger to jump around, and c-c c-w c-c to track who calls the different functions and methods to browse around. I think we need to make one issue tracking this instead, but I'm not looking to make life more difficult for the maintainer.

Wed 5 Aug 2015 20:09:06 BST

I'd suggest closing issue 23.

Wow - slime makes such a world of difference!

My interpretation:

buildapp initially used sbcl only and was later
extended to include ccl.

The (make) LISP variable was introduced to select
between the two installed implementations. Unlike
sbcl, ccl has two flavours: 32 and 64 bit (ccl,
ccl64 respectively).

buildapp::build-buildapp expects to find standard
installations of sbcl/ccl ie the compiler located
under its standard name in the user's $PATH.

Moving forward:

1. Modify the Makefile so that LISP can only take
   one of two/three values: sbcl, ccl, ccl64
2. It may be the case that ccl sets a separate
   read-time conditional for ccl64 (#+ccl64).

If so, this can be added to the code to set the
compiler to ccl64.

OR

1. Use the LISP variable as path to lisp compiler.

It should also be possible to augment the code to
set the dumper ccl (and sbcl) to use the value
passed to the make LISP variable.

In the interest of clarity and keeping with the
original intent of buildapp, 1,2 above should be
considered.

---

Reply to this email directly or view it on GitHub:
#24 (comment)

-primus (Train yourself and be your own master.)
"First solve the problem, then code!"
Narrowness of experience leads to narrowness of imagination.
http://www.catb.org/esr/faqs/smart-questions.html
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

GPG Key: DB3FB476
Key fingerprint: B0FB C67E 2E7E 7032 7FE6 7FBC 28E9 2848 DB3F B476

[xach]

I don't like how the ccl support turned out. Buildapp should be able to build sbcl or ccl binaries regardless of which was used to build buildapp itself. And it should be easy to specify which binary is used to do the building, and what type of lisp it is. And it should support more than just SBCL and CCL. To that end, I've done sporadic work redesigning the multi-Lisp support of buildapp. It's in the "xp" branch on github. The general idea is that the dumper file uses functions in the BUILDAPP-SYS package to do all platform-specific work. The code implementing BUILDAPP-SYS functions are stored as text in the buildapp binary and written out at the start of the dumper binary. That way, there is no issue with symbol management. This branch is incomplete. I don't think it would take much more work to finish it, but I've been working on other things and haven't had the time to clean it up and finish it.