[Possible Bug] Does Not Create HTML File of Complete Folder (Posssibly Due to Size Limit)

[OnyxGhub]

When attempting to create a HTML file of one of my drive directories, a HTML file with only the first 5 sub-folders (and all their nested folders) was created when the root directory contains 21 sub-folders. Could this be due to possible HTML file size limit?

Here is a snapshot of the of what the Snap2HTML app gathered for the HTML file (size 67.4 MB), showing 502,218 files in 197,808 folders (135.91 GB).
I show the properties of the first 5 sub-folders shows similar stats: 520,122 files in 206,491 folders (150 GB).
I also show the properties of all 21 sub-directories which is what the HTML file should contain in total: 676,087 files in 216,610 folders (12.6 TB).
https://i.imgur.com/SM2O8U5.png

I can also say that all other 8 drives that I created a HTML file for seemed to be successful at including all files and folders within the root directory and they ranged in size 3.66 MB [50,412 files in 3,447 folders (12.96 TB)] to 18.6 MB [275,339 files in 36,096 folders (1.40 TB)].

If this is a bug due to HTML file size limitations, can it either be fixed to include all of what's in the root directory, or if not, can it be broken out into multiple HTML files once the the first file gets to be too large?

Thanks

[rlv-dan]

It's very hard to tell what could be the problem here since I do not have access to your computer. I must ask you to try and figure out if there is some kind of common denominator with the folders that could give a hint to why they are not included. Make sure that you have include hidden/system checkboxes enabled. Also run as admin just to be sure.

[OnyxGhub]

In every single attempt the "Include hidden items" & "Include system items" options were both enabled. The only thing that sets this drive having issues apart from the other 8 drives which were completed successfully is the much larger amount of total files (518,459) and sub-folders (206,432) being processed by your application which would thus lead to a much larger as shown here:
https://i.imgur.com/cxVzpR6.png
The drive contains a backup directory created by the application Cobian and it stores up to 111 GB of backed up data there. This is the main directory which is leading to the much larger number of files and sub-directories. If the Snap2HTML application allowed me to exclude this one folder from the process, that would be a good test to see if it is indeed where the issue is coming from.

I just tried it again running your application as admin two more times with the following results:

1st attempt (run as admin) with "Include hidden items" & "Include system items" enabled, and "Link files" option enabled: same result as on the 7/13/2022 with the exact same number of of files, folders and data size reported. [HTML file size 67.4 MB, showing 502,218 files in 197,808 folders (135.91 GB)].

2nd attempt (run as admin) with "Include hidden items" & "Include system items" enabled, and "Link files" option disabled: same result as on the 7/13/2022 with the exact same number of of files, folders and data size reported. [HTML file size 67.4 MB, showing 502,218 files in 197,808 folders (135.91 GB)].

The each attempt is giving me the same result which leads me to believe it could be an issue with the total number of files and sub-folders it has to process and include in the HTML file and it could be possibly reaching its limit somehow that way. I'll add that the HTML file contains a total of 199,587 lines of code and 70,516,605 characters.

If you have any other suggestions, I can try those as well. Thank you.

[rlv-dan]

There should be no limitations on the amount of files. ½ million files is not too much.
Right now I don't have a good solution or suggestion on how to continue. But please continue to try to figure out if there is something special with the problematic folders.

[Jarvis-3-0]

I've had a similar problem with files missing from the listings in the generated html. The missing files name contain special characters, which I believe must not be able to be shown in the html. Instead of replacing the character with a ? or similar, it appears that the file just does not make it into the html at all, and hence an incomplete list.
(Correcting) ->I've just double checked the html file in notepad++, and the file names are missing from the D.p line. The examples I'm looking at both contain an ( ' ) in the file name.
I may be able to send such a file that creates an issue if that is of help?

[rlv-dan]

Yes, please send me such a file. Zip it first (if you can - Windows zip will probably not allow it - use 7zip instead). Also paste some filenames here for reference

[Jarvis-3-0]

Thanks. I have ended up spending some extra time looking deeper into this as I was struggling to retrieve the files to send as a zip.

I have 2 copies of the files as they were done on different dates as a snapshot of that folder at that time. The snapshots are both stored on the same nas drive. Snap2HTML was used on this drive after mapping it as a network share from windows. Upon further investigation I have found that one copy of the files does not show every file when browsing the mapped drive, although if I use winscp to view the drive, all the files are correct and present. So apologies, as it looks like this is not a Snap2HTML issue and I was confusing myself comparing different versions of the same folders and files.

However, the below may be of benefit to troubleshooting the original posters issue or future issues someone else may have. As the files are stored on a Linux based nas, I have ended up using convmv to inspect the files with issues. It appears that the file (and folder) names with issues are encoded in cp1252 instead of utf-8. Winscp is able to cope with this, but many other programs have issues, ranging from file names with ? where it does not recognise the character, to files being ignored or programs hanging. If I copy the files across using winscp, it re-encodes the file names, so the issue disappears.

[rlv-dan]

Interesting. I will need to investigate this further.

[Left-Handed-Cat]

@OnyxGhub I would check, if there are very long folder pathes incl. filenames exceeding the 255 character limit. This problem is already reported and can abort the processing. The result is an incomplete html file. Check this: #16