From 989358de6ef68e75c6dd964c939e29d03192012f Mon Sep 17 00:00:00 2001 From: "Fletcher T. Penney" Date: Wed, 15 Mar 2017 11:52:49 -0400 Subject: [PATCH 01/32] FIXED: Fix Fuzz LaTeX test --- tests/MMD6Tests/Fuzz.tex | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/MMD6Tests/Fuzz.tex b/tests/MMD6Tests/Fuzz.tex index e0ba52dd..aec37101 100644 --- a/tests/MMD6Tests/Fuzz.tex +++ b/tests/MMD6Tests/Fuzz.tex @@ -6,7 +6,7 @@ Collection of test cases identified by \href{http://lcamtuf.coredump.cx/afl/}{American fuzzy lop}\footnote{\href{http://lcamtuf.coredump.cx/afl/}{http:\slash \slash lcamtuf.coredump.cx\slash afl\slash }}. ->bar~~} +>bar~~\} \begin{itemize} \item{} list From 10421bd50fe29f486afe95d60fde068dd8456a5d Mon Sep 17 00:00:00 2001 From: "Fletcher T. Penney" Date: Wed, 15 Mar 2017 12:33:05 -0400 Subject: [PATCH 02/32] CHANGED: Update license and development notes --- DevelopmentNotes/DevelopmentNotes.txt | 43 +++++++++++++++++++++++---- LICENSE.txt | 38 ++++++++++++++++++++++- README.md | 38 ++++++++++++++++++++++- 3 files changed, 112 insertions(+), 7 deletions(-) diff --git a/DevelopmentNotes/DevelopmentNotes.txt b/DevelopmentNotes/DevelopmentNotes.txt index da5c2b23..9d9ee451 100644 --- a/DevelopmentNotes/DevelopmentNotes.txt +++ b/DevelopmentNotes/DevelopmentNotes.txt @@ -187,8 +187,7 @@ this quickly fell apart in the face of more complex structures such as recursive lists. After a lot of searching, and *tons* more reading, I ultimately decided to use a parser generator to handle the task of group lines into blocks. `parser.y` has the source for this, and it is processed by the -[lemon](http://www.hwaci.com/sw/lemon/) parser generator to create the actual -code. +[lemon] parser generator to create the actual code. I chose to do this because hand-crafting the block parser would be complex. The end result would likely be difficult to read and understand, which would @@ -419,16 +418,50 @@ the development of MMD. So there is really not much unit testing built into the code or used during the development. +# Dependencies/Libraries # + +MMD v6 has no external dependencies when compiling, aside from the standard +libraries for C development. + +MMD can be compiled without any other tools beside the build system (cmake). + +If you want to edit the block parser, you need to modify the `parser.y` file +and process that using [lemon] in order to update the `parser.c` file. The +lemon parser source is actually included in MMD and needs to be compiled to be +used. + +If you want to update the lexer or scanner utility functions, then you can +modify `lexer.re` or `scanners.re`. These need to be processed using [re2c], +which has to be installed separately. + +MMD v6 makes use of several other projects to improve performance and ease of +use: + +* [uthash] -- provides support for hashes to quickly locate specific "objects" +based on text keys. These are used for matching footnotes, links, images and +the like when created with the reference syntax. Much faster than searching +through each one sequentially. + +* [miniz] -- provides zip archive support to enable creation of EPUB 3 files, +which are zip files with a specific file structure inside them. + +* [argtable3] -- provides more advanced command-line argument processing in a +more cross-platform approach than that used by MMD v5. + [>MMD]: MultiMarkdown [>MD]: Markdown -[CriticMarkup]: http://criticmarkup.com/ - [?PEG]: Parsing Expression Grammar - [?AST]: Abstract Syntax Tree +[CriticMarkup]: http://criticmarkup.com/ +[lemon]: http://www.hwaci.com/sw/lemon/ +[re2c]: http://re2c.org/ +[uthash]: https://troydhanson.github.io/uthash/ +[miniz]: https://github.com/richgel999/miniz +[argtable3]: https://github.com/argtable/argtable3 + # Changelog # diff --git a/LICENSE.txt b/LICENSE.txt index c10f45ea..82ccae9d 100644 --- a/LICENSE.txt +++ b/LICENSE.txt @@ -13,7 +13,7 @@ text of the license. uthash library: Copyright (c) 2005-2016, Troy D. Hanson - Licensed under BSD Revised license + Licensed under Revised BSD license miniz library: Copyright 2013-2014 RAD Game Tools and Valve Software @@ -21,6 +21,13 @@ miniz library: Licensed under the MIT license +argtable3 library: + Copyright (C) 1998-2001,2003-2011,2013 Stewart Heitmann + + All rights reserved. + + Licensed under the Revised BSD License + ## The MIT License ## @@ -42,3 +49,32 @@ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + +## Revised BSD License ## + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials provided + with the distribution. + * Neither the name of the nor the + names of its contributors may be used to endorse or promote + products derived from this software without specific prior + written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, +EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR +PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF +LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING +NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/README.md b/README.md index b7fa26cd..d8374157 100644 --- a/README.md +++ b/README.md @@ -137,7 +137,7 @@ information. uthash library: Copyright (c) 2005-2016, Troy D. Hanson - Licensed under BSD Revised license + Licensed under Revised BSD license miniz library: Copyright 2013-2014 RAD Game Tools and Valve Software @@ -145,6 +145,13 @@ information. Licensed under the MIT license + argtable3 library: + Copyright (C) 1998-2001,2003-2011,2013 Stewart Heitmann + + All rights reserved. + + Licensed under the Revised BSD License + ## The MIT License ## @@ -167,3 +174,32 @@ information. TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + ## Revised BSD License ## + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions are + met: + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials provided + with the distribution. + * Neither the name of the nor the + names of its contributors may be used to endorse or promote + products derived from this software without specific prior + written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, + EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, + PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE, DATA, OR + PROFITS OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + From cee506300c8e0c1e9a7b1f1344d3fa5bddfd1e3b Mon Sep 17 00:00:00 2001 From: "Fletcher T. Penney" Date: Wed, 15 Mar 2017 14:20:46 -0400 Subject: [PATCH 03/32] ADDED: Add mmd2epub convenience script --- CMakeLists.txt | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 54cd14d9..2dfb004f 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -227,8 +227,9 @@ set(header_files set (scripts scripts/mmd scripts/mmd2all - scripts/mmd2opml + scripts/mmd2epub scripts/mmd2odf + scripts/mmd2opml scripts/mmd2pdf scripts/mmd2tex scripts/markdown @@ -508,7 +509,7 @@ endif (DEFINED SHAREDBUILD) # Install LaTeX support files install (FILES ${latex} - DESTINATION share/texmf/mmd6 + DESTINATION share/texmf/tex/latex/mmd6 COMPONENT latex PERMISSIONS OWNER_WRITE OWNER_READ GROUP_READ WORLD_READ ) From 8846dd9284067563f5311a98fb8e699e70788421 Mon Sep 17 00:00:00 2001 From: "Fletcher T. Penney" Date: Wed, 15 Mar 2017 14:21:14 -0400 Subject: [PATCH 04/32] ADDED: Add mmd2epub convenience script --- scripts/mmd2epub | 29 +++++++++++++++++++++++++++++ scripts/mmd2epub.bat | 8 ++++++++ 2 files changed, 37 insertions(+) create mode 100755 scripts/mmd2epub create mode 100755 scripts/mmd2epub.bat diff --git a/scripts/mmd2epub b/scripts/mmd2epub new file mode 100755 index 00000000..11f77976 --- /dev/null +++ b/scripts/mmd2epub @@ -0,0 +1,29 @@ +#!/bin/sh +# +# mmd2oepub --- MultiMarkdown convenience script +# +# Fletcher T. Penney +# +# Pass arguments on to the binary to convert text to EPUB 3 +# + +# Be sure to include multimarkdown in our PATH +export PATH="$PWD:/usr/local/bin:$PATH" + +which multimarkdown > /dev/null +if [ $? = 1 ] +then + echo multimarkdown executable not found! >&2 + exit 1 +fi + +if [ $# = 0 ] +then + multimarkdown -t epub +else +until [ "$*" = "" ] +do + multimarkdown -b -t epub "$1" + shift +done +fi diff --git a/scripts/mmd2epub.bat b/scripts/mmd2epub.bat new file mode 100755 index 00000000..36ba3737 --- /dev/null +++ b/scripts/mmd2epub.bat @@ -0,0 +1,8 @@ +@ECHO OFF +:Loop +IF "%1"=="" GOTO Continue + "%~dp0\multimarkdown" -b -t epub "%1" +SHIFT +GOTO Loop +:Continue + From ef1a53a3ddcce8b21a86cab58e18fb5a92716261 Mon Sep 17 00:00:00 2001 From: "Fletcher T. Penney" Date: Wed, 15 Mar 2017 14:21:30 -0400 Subject: [PATCH 05/32] ADDED: Add mmd2epub convenience script --- scripts/mmd2all | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/scripts/mmd2all b/scripts/mmd2all index fe0294cb..94b89875 100755 --- a/scripts/mmd2all +++ b/scripts/mmd2all @@ -29,13 +29,15 @@ do multimarkdown -b "$1" # open "$file_name.html" + multimarkdown -b -t epub "$1" + multimarkdown -b -t latex "$1" # mate "$file_name.tex" multimarkdown -b -t odf "$1" # open "$file_name.fodt" - multimarkdown -b -t opml "$1" +# multimarkdown -b -t opml "$1" # open "$file_name.opml" shift From 5bf59641ac943e944daf8b73217f3e56b2161853 Mon Sep 17 00:00:00 2001 From: "Fletcher T. Penney" Date: Wed, 15 Mar 2017 14:32:29 -0400 Subject: [PATCH 06/32] FIXED: Fix potential crash --- Sources/libMultiMarkdown/writer.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Sources/libMultiMarkdown/writer.c b/Sources/libMultiMarkdown/writer.c index d547eb5d..1e882f6d 100644 --- a/Sources/libMultiMarkdown/writer.c +++ b/Sources/libMultiMarkdown/writer.c @@ -2211,7 +2211,8 @@ void strip_leading_whitespace(token * chain, const char * source) { return; } - chain = chain->next; + if (chain) + chain = chain->next; } } From 27786eb78c8fae16306ec1b22229661285c446e3 Mon Sep 17 00:00:00 2001 From: "Fletcher T. Penney" Date: Wed, 15 Mar 2017 14:33:33 -0400 Subject: [PATCH 07/32] FIXED: Fix potential crash --- Sources/libMultiMarkdown/writer.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Sources/libMultiMarkdown/writer.c b/Sources/libMultiMarkdown/writer.c index 1e882f6d..2e75fc82 100644 --- a/Sources/libMultiMarkdown/writer.c +++ b/Sources/libMultiMarkdown/writer.c @@ -1304,7 +1304,8 @@ void process_definition_block(mmd_engine * e, token * block) { break; } label->type = TEXT_EMPTY; - label->next->type = TEXT_EMPTY; + if (label->next) + label->next->type = TEXT_EMPTY; strip_leading_whitespace(label, e->dstr->str); break; case BLOCK_DEF_LINK: From d4344411f5e9d08ddbae9ba5e56881f319de0c2c Mon Sep 17 00:00:00 2001 From: "Fletcher T. Penney" Date: Wed, 15 Mar 2017 14:58:45 -0400 Subject: [PATCH 08/32] UPDATE: Update documentation --- DevelopmentNotes/DevelopmentNotes.fodt | 39 +++++++++++++++++++++++-- DevelopmentNotes/DevelopmentNotes.html | 33 +++++++++++++++++++-- DevelopmentNotes/DevelopmentNotes.pdf | Bin 105192 -> 107594 bytes QuickStart/QuickStart.fodt | 5 ---- QuickStart/QuickStart.html | 7 ++--- QuickStart/QuickStart.pdf | Bin 89792 -> 89103 bytes QuickStart/QuickStart.txt | 11 ------- 7 files changed, 70 insertions(+), 25 deletions(-) diff --git a/DevelopmentNotes/DevelopmentNotes.fodt b/DevelopmentNotes/DevelopmentNotes.fodt index f02814d3..130065e3 100644 --- a/DevelopmentNotes/DevelopmentNotes.fodt +++ b/DevelopmentNotes/DevelopmentNotes.fodt @@ -501,8 +501,7 @@ this quickly fell apart in the face of more complex structures such as recursive lists. After a lot of searching, and tons more reading, I ultimately decided to use a parser generator to handle the task of group lines into blocks. parser.y has the source for this, and it is processed by the -lemon parser generator to create the actual -code. +lemon parser generator to create the actual code. I chose to do this because hand-crafting the block parser would be complex. The end result would likely be difficult to read and understand, which would @@ -716,6 +715,42 @@ Otherwise, it was hard to see how to really create very good unit tests for the development of MMD. So there is really not much unit testing built into the code or used during the development. +Dependencies/Libraries + +MMD v6 has no external dependencies when compiling, aside from the standard +libraries for C development. + +MMD can be compiled without any other tools beside the build system (cmake). + +If you want to edit the block parser, you need to modify the parser.y file +and process that using lemon in order to update the parser.c file. The +lemon parser source is actually included in MMD and needs to be compiled to be +used. + +If you want to update the lexer or scanner utility functions, then you can +modify lexer.re or scanners.re. These need to be processed using re2c, +which has to be installed separately. + +MMD v6 makes use of several other projects to improve performance and ease of +use: + + + +uthash – provides support for hashes to quickly locate specific “objects” +based on text keys. These are used for matching footnotes, links, images and +the like when created with the reference syntax. Much faster than searching +through each one sequentially. + + +miniz – provides zip archive support to enable creation of EPUB 3 files, +which are zip files with a specific file structure inside them. + + +argtable3 – provides more advanced command-line argument processing in a +more cross-platform approach than that used by MMD v5. + + + Changelog diff --git a/DevelopmentNotes/DevelopmentNotes.html b/DevelopmentNotes/DevelopmentNotes.html index b7edd230..2fcee042 100644 --- a/DevelopmentNotes/DevelopmentNotes.html +++ b/DevelopmentNotes/DevelopmentNotes.html @@ -181,8 +181,7 @@

Second Attempt

recursive lists. After a lot of searching, and tons more reading, I ultimately decided to use a parser generator to handle the task of group lines into blocks. parser.y has the source for this, and it is processed by the -lemon parser generator to create the actual -code.

+lemon parser generator to create the actual code.

I chose to do this because hand-crafting the block parser would be complex. The end result would likely be difficult to read and understand, which would @@ -396,6 +395,36 @@

Unit Testing

the development of MMD. So there is really not much unit testing built into the code or used during the development.

+

Dependencies/Libraries

+ +

MMD v6 has no external dependencies when compiling, aside from the standard +libraries for C development.

+ +

MMD can be compiled without any other tools beside the build system (cmake).

+ +

If you want to edit the block parser, you need to modify the parser.y file +and process that using lemon in order to update the parser.c file. The +lemon parser source is actually included in MMD and needs to be compiled to be +used.

+ +

If you want to update the lexer or scanner utility functions, then you can +modify lexer.re or scanners.re. These need to be processed using re2c, +which has to be installed separately.

+ +

MMD v6 makes use of several other projects to improve performance and ease of +use:

+ +
    +

  • uthash – provides support for hashes to quickly locate specific “objects” +based on text keys. These are used for matching footnotes, links, images and +the like when created with the reference syntax. Much faster than searching +through each one sequentially.

    • +

  • miniz – provides zip archive support to enable creation of EPUB 3 files, +which are zip files with a specific file structure inside them.

    • +

  • argtable3 – provides more advanced command-line argument processing in a +more cross-platform approach than that used by MMD v5.

    • +
+

Changelog