How to Serve a Single TW5 "Export all Tiddlers" (HTML Extension) File? #9

ChintiminiFarm · 2017-11-26T23:04:14Z

Hi Danielo:

Thanks for the effort and expertise you have put into this new project! Can this system support the following use case?

I would like to upload a single TW5 file (named: "my-new-project.html") into the project space (in whatever appropriate spot is best) and configure it (with appropriate syntax) in the "tiddlywiki.info" file.

Details:

When a new user (me) wants to try TW5, I download a new empty wiki file.
I end up with a single HTML file, that I can name to something like: 'my-new-project.html".
(Well, we know it's not "just" an HTML file, it's also whole lot of JavaScript "magic dust" as well).
I get busy & make a bunch of new tiddlers, maybe hundreds. But to naive user (me) it's still a single HTML file.
When I'm done, I would like to upload "my-new-project.html" to my own fork (of your project) and have it served from there.
But I see the current build configuration seems to be based on .tid files.
And if I try to "export all tiddlers" into a tid file, I find out that TW5 only exports a single tiddler.

So, am I stuck? Or is there a way to configure a Travis CI "build" for a single html file?

The best of all possible worlds for me would be to upload my completed TW5 wiki as a single HTML file into the wiki directory, edit one or more config files to say: "look for this target HTML file", let Travis CI detect the push event and have the auto-config "magic" do its stuff.

Best regards,

-- Daniel

danielo515 · 2017-11-27T16:09:10Z

Hello @ChintiminiFarm
Travis CI can do whatever you want. This repository was one of my first approaches to Travis CI, so there is room for improvement.

Your question is more related to tiddlywiki than Travis CI. Travis is just a task executor, and in this case we are using it to execute several tiddlywiki commands. As far as I know, tiddlywiki supports importing tiddlers from a single file, so your scenario should be possible. In fact, with the correct configuration I think that (not sure) you can support both .tid files and importing files from a single wiki.

danielo515 · 2017-11-27T16:13:38Z

The load command should allow you to extract the tiddlers contained on your my-new-project.html file and then continue with the rest of the build normally.

Please give it a try and let me know your findings.

ChintiminiFarm · 2017-11-28T07:19:35Z

Thanks for the recommendation, Danielo. Here are my results:

Here is my branch: feature-generate-from-single-html-file from a more pristine fork of your original project (since I had slopped the first try up into a big mess).

I experimented by loading a generic TW5 wiki: wiki.explore.gardening.html into the the .wiki directory (from the Travis CI build logfiles, that seemed to be the place to put it). This is a known good TW5 file, not too large, with generic tiddler content.

I tried variations of the load command both (1) in the travis.yml file:

script:
  - tiddlywiki ./*wiki --load wiki.explore.gardening.html
  - tiddlywiki ./*wiki --build

and (2) in the tiddlywiki.info file:

	"build": {
		"index": [
			"--load","wiki_explore_gardening.html",
			"--rendertiddler","$:/plugins/tiddlywiki/tiddlyweb/save/offline","index.html","text/plain"],

Unfortunately, the --load commands failed in both instances. Here is output from Travis CI:

4.42s$ npm install -g tiddlywiki
/home/travis/.nvm/versions/node/v9.2.0/bin/tiddlywiki -> /home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/tiddlywiki.js

+ [email protected]
added 1 package in 3.968s
0.57s$ tiddlywiki ./*wiki --load wiki.explore.gardening.html

Error: No tiddlers found in file "wiki.explore.gardening.html"

The command "tiddlywiki ./*wiki --load wiki.explore.gardening.html" exited with 1.

0.64s$ tiddlywiki ./*wiki --build
Error: No tiddlers found in file "wiki_explore_gardening.html"

The command "tiddlywiki ./*wiki --build" exited with 1.
Done. Your build exited with 1.

Searching on the forums for this error has yielded no results.
It's a bummer, because I was excited by your idea for a solution.
Any ideas? Perhaps a simple syntax error on my part?

Best regards,
-- Daniel

danielo515 · 2017-11-28T10:31:33Z

Hello @ChintiminiFarm
Looking at your setup I think your error resides on the path or path names.
The wiki folder should only contain tiddler files and the tiddlywiki.info file.
Try this:

move your html file to the root of the repository
execute this command tiddlywiki ./wiki --load ./wiki.explore.gardening.html
execute tiddlywiki ./wiki --build

That should do the trick. What we are doing is first importing all the tiddlers that your wiki contains into the wiki folder and then build the wiki using them.

Give it a try and let me know if it solves your problems

ChintiminiFarm · 2017-11-28T22:01:26Z

Hi Danielo -- Again, many thanks for your time, energy & attention on this!

Your recommendations make perfect sense (In past lives, I've dealt with a lot of makefiles). I reconfigured as follows:

I moved the single html file: wiki_explore_gardening.html into the repository root. (I even renamed the file with underbars -- just in case there was a silly error parsing dots before a filename extension. And of course there wasn't)

I kept the tiddlywiki ./wiki --load ./wiki_explore_gardening.html command in the travis.yml file. The log file shows that it is getting properly invoked there. I also removed that sequence from the tiddlywiki.info file "build": section, so we can focus on the sequence with the single invocation that's not working.

So now the sequence should fire off the tiddlywiki --load (from the travis,yml file) and then invoke the tiddlywiki --build (from the tiddlywiki.info file).

We still error out with:

$ nvm install node
Downloading and installing node v9.2.0...
Now using node v9.2.0 (npm v5.5.1)
$ node --version
v9.2.0
$ npm --version
5.5.1
$ nvm --version
0.33.6
before_script

10.48s$ npm install -g tiddlywiki
/home/travis/.nvm/versions/node/v9.2.0/bin/tiddlywiki -> /home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/tiddlywiki.js

+ [email protected]
added 1 package in 9.604s

1.27s$ tiddlywiki ./wiki --load wiki_explore_gardening.html

Error: No tiddlers found in file "wiki_explore_gardening.html"
The command "tiddlywiki ./wiki --load wiki_explore_gardening.html" exited with 1.0m

2.56s$ tiddlywiki ./wiki --build
 syncer-server-filesystem: Dispatching 'save' task: $:/StoryList 

For $:/StoryList, type is application/x-tiddler hasMetaFile is false filepath is /home/travis/build/ChintiminiFarm/auto-publish-single-TW5-html-file-on-GH/wiki/tiddlers/$__StoryList.tid

 filesystem: Saved file /home/travis/build/ChintiminiFarm/auto-publish-single-TW5-html-file-on-GH/wiki/tiddlers/$__StoryList.tid 

The command "tiddlywiki ./wiki --build" exited with 0.

Done. Your build exited with 1.

Do you think I'm missing anything else obvious?

Do you think this is enough to open a bug issue with the main TW5 project (I don't want to bother Jeremy if I'm just doing something silly)?

Best regards,
-- Daniel

danielo515 · 2017-11-29T06:38:33Z

Hello, it may be a big with 5.1.15 version, because it works on my machine with .14
Try updating the Travis file this way:

npm install -g [email protected]

If that works we will know it is a bug with the new version and you can open a bug report there.
Regards

ChintiminiFarm · 2017-11-29T08:47:28Z

Hi Danielo -- great troubleshooting idea! Here are my (very interesting) results.

TW5 version 5.1.1.4 gave MUCH better error messages from the build back to Travis CI:

4.24s$ npm install -g [email protected]

/home/travis/.nvm/versions/node/v9.2.0/bin/tiddlywiki -> /home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/tiddlywiki.js

+ [email protected]

added 1 package in 3.78s

0.50s$ tiddlywiki ./wiki --load wiki_explore_gardening.html

fs.js:968

  binding.stat(pathModule.toNamespacedPath(path));

          ^

Error: ENOENT: no such file or directory, stat 'wiki_explore_gardening.html'

    at Object.fs.statSync (fs.js:968:11)

    at Command.execute ($:/core/modules/commands/load.js:34:13)

    at Commander.executeNextCommand ($:/core/modules/commander.js:115:14)

    at Commander.execute ($:/core/modules/commander.js:64:7)

    at Object.exports.startup ($:/core/modules/startup/commands.js:34:12)

    at Object.$tw.boot.executeNextStartupTask (/home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/boot/boot.js:2069:10)

    at Object.$tw.boot.executeNextStartupTask (/home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/boot/boot.js:2067:21)

    at Object.$tw.boot.executeNextStartupTask (/home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/boot/boot.js:2067:21)

    at Object.$tw.boot.executeNextStartupTask (/home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/boot/boot.js:2067:21)

    at Object.$tw.boot.executeNextStartupTask (/home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/boot/boot.js:2067:21)

The command "tiddlywiki ./wiki --load wiki_explore_gardening.html" exited with 1.

1.58s$ tiddlywiki ./wiki --build

syncer-server-filesystem: Dispatching 'save' task: $:/StoryList

For $:/StoryList, type is application/x-tiddler hasMetaFile is false filepath is /home/travis/build/ChintiminiFarm/auto-publish-single-TW5-html-file-on-GH/wiki/tiddlers/$__StoryList.tid

FileSystem: Saved file /home/travis/build/ChintiminiFarm/auto-publish-single-TW5-html-file-on-GH/wiki/tiddlers/$__StoryList.tid

The command "tiddlywiki ./wiki --build" exited with 0.
Done. Your build exited with 1.

So I see it's a stat error, like it can't find the file. How strange.

To troubleshoot, I also uploaded the same (generic TW5) file: wiki_explore_gardening.html into first the the ./wiki subdirectory, and then the ./wiki/tiddlers subdirectory. No change, it still failed with a stat error.

It seems Travis CI, working through the .travis.yml file, and then the TW5 --load command, can not see the source file.

Do you think there is some implicit $PATH variable that needs to be taken into account?

I'm at a loss to say why that would be the case. This seems to be a pretty simple configuration. There's just the repository project directory, and the ./wiki/... tree below, right?

Thanks for your continued efforts (and great troubleshooting ideas) on this, Danielo!

Best regards,

-- Daniel

danielo515 · 2017-11-29T10:34:01Z

Hello @ChintiminiFarm
The output of the error is correct because you are on an environment where you are forced to specify the complete paths. This means that you can not reference a file by just it's name (wiki_explore_gardening.html) you need to provide a path. In this case, the current folder ./wiki_explore_gardening.html so your complete command, as I have already posted must be the following

tiddlywiki ./wiki --load ./wiki_explore_gardening.html

And the html file must be placed at the root of the directory. I'm very sure that this should work.
Regards

ChintiminiFarm · 2017-11-29T11:09:53Z

Hi Danielo,

I did try that test case earlier, just to try to troubleshoot, and it didn't work for me. To be sure though, I just did it again and here are the results (there is interesting stuff going on with the $PATH variable):

Setting environment variables from repository settings
$ export GH_TOKEN=[secure]
$ export GH_EMAIL=[secure]
$ export GH_REF=github.com/ChintiminiFarm/auto-publish-single-TW5-html-file-on-GH.git

$ export PATH=./node_modules/.bin:$PATH

Updating nvm
nvm.install

3.44s$ nvm install node
Downloading and installing node v9.2.0...
Downloading https://nodejs.org/dist/v9.2.0/node-v9.2.0-linux-x64.tar.xz...
######################################################################## 100.0%

Computing checksum with sha256sum
Checksums matched!
Now using node v9.2.0 (npm v5.5.1)

$ node --version
v9.2.0

$ npm --version
5.5.1

$ nvm --version
0.33.6
before_script

6.28s$ npm install -g [email protected]

/home/travis/.nvm/versions/node/v9.2.0/bin/tiddlywiki -> /home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/tiddlywiki.js

+ [email protected]
added 1 package in 5.575s

0.70s$ tiddlywiki ./wiki --load ./wiki_explore_gardening.html
fs.js:968
  binding.stat(pathModule.toNamespacedPath(path));
          ^
Error: ENOENT: no such file or directory, stat './wiki_explore_gardening.html'
    at Object.fs.statSync (fs.js:968:11)
    at Command.execute ($:/core/modules/commands/load.js:34:13)
    at Commander.executeNextCommand ($:/core/modules/commander.js:115:14)
    at Commander.execute ($:/core/modules/commander.js:64:7)
    at Object.exports.startup ($:/core/modules/startup/commands.js:34:12)
    at Object.$tw.boot.executeNextStartupTask (/home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/boot/boot.js:2069:10)
    at Object.$tw.boot.executeNextStartupTask (/home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/boot/boot.js:2067:21)
    at Object.$tw.boot.executeNextStartupTask (/home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/boot/boot.js:2067:21)
    at Object.$tw.boot.executeNextStartupTask (/home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/boot/boot.js:2067:21)
    at Object.$tw.boot.executeNextStartupTask (/home/travis/.nvm/versions/node/v9.2.0/lib/node_modules/tiddlywiki/boot/boot.js:2067:21)

The command "tiddlywiki ./wiki --load ./wiki_explore_gardening.html" exited with 1.

1.69s$ tiddlywiki ./wiki --build
syncer-server-filesystem: Dispatching 'save' task: $:/StoryList

For $:/StoryList, type is application/x-tiddler hasMetaFile is false filepath is /home/travis/build/ChintiminiFarm/auto-publish-single-TW5-html-file-on-GH/wiki/tiddlers/$__StoryList.tid

FileSystem: Saved file /home/travis/build/ChintiminiFarm/auto-publish-single-TW5-html-file-on-GH/wiki/tiddlers/$__StoryList.tid

The command "tiddlywiki ./wiki --build" exited with 0.
Done. Your build exited with 1.

So... I noticed with your original setup instructions, we setup environment variables GH_TOKEN, GH_REF, and GH_EMAIL. Perhaps we need to do something additional with $PATH as well?

The truly strange thing is that the source file: ./wiki_explore_gardening.html now has an explicit path designator "./", and the source file exists at all levels of the project directory structure (because it's been redundantly duplicated). And for sure, as you recommend, it's at the root of the project directory. And we can see it is in fact getting invoked with that path designator. So you think that wherever the build commands think they have "come to life", they should be able to see one of the (duplicated) instances of the source wiki file. But it misses them all. Truly bizarre!

Thanks for the continued assistance. I'm going to crash for a while. It's 03:00 on the Pacific Coast. Where are you?

Best regards,

-- Daniel

danielo515 · 2017-11-29T11:31:32Z

Hello @ChintiminiFarm
I'm on Madrid (Spain) which is on europe. It's 12 in the morning right now.

I'm not sure if you have noticed this, but there is an space on your file name ./wiki_explore_gardening .html
Note the space between the name and the extension

danielo515 · 2017-11-29T13:27:56Z

Hello @ChintiminiFarm
I have created a sample branch that uses your example html file and does exactly what you want:

https://github.com/danielo515/TW5-auto-publish2gh-pages/tree/feature/buildFromFile

Regards

ChintiminiFarm · 2017-11-29T19:55:50Z

Thank you, Danielo!

Heavy sigh. Hijo de (CENSORED)!!!! Curse non-monospaced fonts in "rename" file management dialog boxes! We do not need "pretty" when we do file management tasks. Arrgh! My error was so far upstream that I never noticed it.

Your eye is good. Thanks for the troubleshooting. I owe you dinner & drinks if you & yours ever come to Portland.

Two proposals for you:

Since this technique now works, I consider it very useful for publishing "already done" wikis that people might want to share. Shall I write up a summary and post to the forum?
Would you like me to update some of your documentation? I can do that and then generate a pull request for you. I noticed some minor typos, and I think that the section when you talk about generating push events needs more explanation. (I found you can't always do that unless you run git on your machine, or artificially generate a push event by file manipulation in the repository. This might be confusing to someone with no exposure to revision control systems that do handshaking with CI systems like Travis).

Again, mucho gracias, amigo. Mucho gracias.

danielo515 · 2017-11-30T07:42:38Z

Heavy sigh. Hijo de (CENSORED)!!!! Curse non-monospaced fonts in "rename" file management dialog boxes!

Hahahaha, you're right! I am not very fan of pretty things regarding computing productivity.

Your eye is good. Thanks for the troubleshooting. I owe you dinner & drinks if you & yours ever come to Portland.

Thank you very much 😄 It was not so hard, but if I go to Portland I will give you a shout.

Since this technique now works, I consider it very useful for publishing "already done" wikis that people might want to share. Shall I write up a summary and post to the forum?

Yes, I also think this may be useful to someone else. It's up to you if you want to share your experience on the forum. I generally consider this a good thing.

Would you like me to update some of your documentation? I can do that and then generate a pull request for you.

That will be the best in my opinion. People will expect to find the documentation here, which makes much more sense along with the fact that it won't be lost in a couple of days like happens on the forum.
If you correct some typos, add some explanations and make clear certain sections that will be an huge contribution.
My original idea with this repository was to remove the need of using a git client. So I want users to be able to just upload an html file and pick it automatically. In any case, you can focus on the documentation part, and I can pick the automation thing later.

Regards

danielo515 · 2017-12-11T12:03:57Z

Hello @ChintiminiFarm can be consider this issue fixed ?
PD: I'll await your PR for further discussion about the implementation.

ChintiminiFarm · 2017-12-18T21:31:33Z

Hi Danielo, Sorry for the delay, Daniel -- holiday madness reigns supreme in this household! Sure, the fix is 100% awesome. As are you! I will schedule some time ASAP to generate something useful in a PR for you. Once again, thanks for the help. This was a great interaction ! Best regards,

…

-- Daniel

On Mon, Dec 11, 2017 at 4:03 AM, Daniel Rodríguez Rivero < ***@***.***> wrote: Hello @ChintiminiFarm <https://github.com/chintiminifarm> can be consider this issue fixed ? PD: I'll await your PR for further discussion about the implementation. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AfoMJ7dVfq7BFcOmmLVMZTsXFZEZOxSNks5s_RougaJpZM4Qq9rG> .

ChintiminiFarm · 2017-12-18T21:39:00Z

Hi Danielo, One quick question for you (for the documentation improvement process): In your personal GitHub/GitLab work-flow, how do you generate a push request? This might be necessary for certain points in the work-flow when you need to "refresh/rebuild". I found myself stuck here a few times, because I did not see any kind of control for a push event from the GitHub web interface. In my case, I knew what was needed, so I "faked it" and "forced" a Travis-CI rebuild by actively updating a source file (I "touched" a file by tweaking a character, and committing. But I knew that was ugly & very clumsy). But I did not see a way of doing that from the GitHub web interface. I surmised you are using an actual git client (or command) on your local machine, correct? Or maybe there's a "push" command interface hidden somewhere in the GitHub UI? Best regards,

…

-- Daniel On Mon, Dec 18, 2017 at 1:31 PM, Daniel Cunningham < [email protected]> wrote:

Hi Danielo, Sorry for the delay, Daniel -- holiday madness reigns supreme in this household! Sure, the fix is 100% awesome. As are you! I will schedule some time ASAP to generate something useful in a PR for you. Once again, thanks for the help. This was a great interaction ! Best regards, -- Daniel On Mon, Dec 11, 2017 at 4:03 AM, Daniel Rodríguez Rivero < ***@***.***> wrote: > Hello @ChintiminiFarm <https://github.com/chintiminifarm> can be > consider this issue fixed ? > PD: I'll await your PR for further discussion about the implementation. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#9 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AfoMJ7dVfq7BFcOmmLVMZTsXFZEZOxSNks5s_RougaJpZM4Qq9rG> > . >

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to Serve a Single TW5 "Export all Tiddlers" (HTML Extension) File? #9

How to Serve a Single TW5 "Export all Tiddlers" (HTML Extension) File? #9

ChintiminiFarm commented Nov 26, 2017

danielo515 commented Nov 27, 2017

danielo515 commented Nov 27, 2017

ChintiminiFarm commented Nov 28, 2017

danielo515 commented Nov 28, 2017

ChintiminiFarm commented Nov 28, 2017 •

edited

Loading

danielo515 commented Nov 29, 2017

ChintiminiFarm commented Nov 29, 2017 •

edited

Loading

danielo515 commented Nov 29, 2017

ChintiminiFarm commented Nov 29, 2017

danielo515 commented Nov 29, 2017

danielo515 commented Nov 29, 2017

ChintiminiFarm commented Nov 29, 2017 •

edited

Loading

danielo515 commented Nov 30, 2017

danielo515 commented Dec 11, 2017

ChintiminiFarm commented Dec 18, 2017 via email

ChintiminiFarm commented Dec 18, 2017 via email

How to Serve a Single TW5 "Export all Tiddlers" (HTML Extension) File? #9

How to Serve a Single TW5 "Export all Tiddlers" (HTML Extension) File? #9

Comments

ChintiminiFarm commented Nov 26, 2017

danielo515 commented Nov 27, 2017

danielo515 commented Nov 27, 2017

ChintiminiFarm commented Nov 28, 2017

danielo515 commented Nov 28, 2017

ChintiminiFarm commented Nov 28, 2017 • edited Loading

danielo515 commented Nov 29, 2017

ChintiminiFarm commented Nov 29, 2017 • edited Loading

danielo515 commented Nov 29, 2017

ChintiminiFarm commented Nov 29, 2017

danielo515 commented Nov 29, 2017

danielo515 commented Nov 29, 2017

ChintiminiFarm commented Nov 29, 2017 • edited Loading

danielo515 commented Nov 30, 2017

danielo515 commented Dec 11, 2017

ChintiminiFarm commented Dec 18, 2017 via email

ChintiminiFarm commented Dec 18, 2017 via email

ChintiminiFarm commented Nov 28, 2017 •

edited

Loading

ChintiminiFarm commented Nov 29, 2017 •

edited

Loading

ChintiminiFarm commented Nov 29, 2017 •

edited

Loading