How to setup the dataset A1 for training #3

Khoa-NT · 2019-10-29T07:49:32Z

Hi,

I read the tutorial but I can't find the location to place the subset A1.
I would like to pre-produce your project follow this tutorial but I don't know where to place the subset A1 ( which is in CVPPP2017_LSC_training\CVPPP2017_LSC_training\training\A1 ) in the directory /code ?

I placed the folder cvppp/A1 in the /code but after I ran ./runs/cvppp_preproc.sh . I only got "nohup: failed to run command 'th': No such file or directory" in the file checkpoints\cvppp_611_preproc\001_train.log
Even with only cvppp/(all files in A1) , I still got the same problem.

Would you mind supporting me about this?

Khoa-NT · 2019-10-31T06:23:38Z

Hi,

I placed the dataset on acis-master\code\data\cvppp\{plantxxx_label.png and plantxxx_rgb.png}

I installed LUA TORCH and now I can run ./runs/cvppp_preproc.sh
But I only got this result

--         Action: 7x7	
Setting the seed 128	
=> Creating model from file: models/preproc.lua	
Checking cache /home/Alexandrite/khoa/Segmentation_RL/CVPR/acis-master/code/gen/cvppp_train.t7	
Loading /home/Alexandrite/khoa/Segmentation_RL/CVPR/acis-master/code/gen/cvppp_train.t7	
Done

I didn't see any logs in "code\checkpoints\cvppp_611_preproc" or in the terminl even there were running tasks of "luajit" at that time.

Should I keep waiting? @arnike

arnike · 2019-10-31T09:03:51Z

Hi @shaolinkhoa,

I apologise for late replies, but I'm currently short of time. The scripts in ./runs are set up in a way that they write logs into a log file rather than output it on screen, so the processes run in the background. The log files are created in ./checkpoints. Also, the models and optimisation state are saved there every few epochs (controlled by the options).

Hope this helps.
Best,
Nikita

Khoa-NT · 2019-10-31T11:13:25Z

Hi @arnike
Thank you for your reply. I was hopeless.

I apologise for late replies, but I'm currently short of time. The scripts in ./runs are set up in a way that they write logs into a log file rather than output it on screen, so the processes run in the background. The log files are created in ./logs directory. In ./checkpoints there should be only models and optimisation state saved every few epochs (controlled by the options).

I'm sorry, I can't find any "models and optimisation state saved " in the checkpoints.
I also created the code/logs directory and ran the ./runs/cvppp_preproc.sh again, but still nothing happend.
The only thing I have is a file acis-master\code\checkpoints\cvppp_611_preproc\001_train.log contains:

Using CVPPP dataset	
--        Dataset: cvppp	
--  learning rate: 0.001	
--        Q-Gamma: 1	
--    Latent Size: 16	
--     Batch Size: 4	
---------------------------------	
{
  height : 530
  loss : "bce"
  checkOnly : false
  model_id : "611_preproc"
  gammaOp : 1
  valTrain : false
  maxNum : 100
  subLastMask : false
  betaOp : 1
  vhigh : false
  actorVer : "simple"
  gpu_id : 1
  numTestRuns : 1
  Size : 224
  invMask : false
  decoder : ""
  logRewardNorm : false
  sampler : "vae"
  sigma : 0
  withBCELoss : false
  preproc_mode : true
  enableContext : false
  nums : 4
  decVer : "simple"
  criticType : "fc"
  actorNoise : 0
  batch_size : 4
  iter_size : 2
  criticLoss : "dice"
  kernelSigmaF : 0.1
  data : "data/cvppp/A1_RAW/"
  learnDecoder : false
  noSampling : false
  actType : "simple"
  actorBN : false
  gen : "gen"
  loadSeqLengthTest : 21
  kernelSigmaL : 1
  dataset : "cvppp"
  learnControl : false
  blockExtMem : false
  maxEpochSize : 256
  pyramidAngles : false
  decVerMod : "none"
  reset_critic : false
  decVerUp : "deconv"
  Crop : 224
  bceCoeff : 1
  numTrainRuns : 1
  save : "checkpoints"
  tensorType : "torch.CudaTensor"
  predictFg : true
  nSamples : 16
  seq_length : 21
  criticMem : 3
  gradClip : 0
  preproc : ""
  learning_rate : 0.001
  withContext : false
  cudnn : "deterministic"
  logReward : false
  noEnc : false
  numAngles : 8
  oldControl : false
  markov : false
  blockHidden : false
  betaDec : 0.1
  max_seq_length : 21
  rnnSize : 256
  nGPU : 1
  withLSTM : false
  loadLastTest : false
  lambdaKL : 1
  manualSeed : 128
  save-stat : false
  optimActor : "adam"
  subGt : false
  learnOp : false
  semSeg : false
  preproc_save : false
  featureSize : 256
  preproc_epoch : 15
  loadSeqLength : 21
  buffer_size : 1024
  growthRate : 32
  ignoreBg : false
  ySize : 237
  xSize : 224
  continue : ""
  criticDim : 32
  imWidth : 224
  crayonPort : 6039
  imHeight : 224
  imageCh : 3
  bottleneck : "fc"
  reduction : 0.5
  optMemory : 3
  lambda : 0
  backend : "cudnn"
  discreteLoss : false
  momentum : 0.9
  width : 500
  loadLast : false
  dropRate : 0
  criticVer : "simple"
  contextDim : 3
  grad : "ac"
  actionX : 7
  contextVer : "simple"
  optimCritic : "adam"
  markovCritic : false
  numClasses : 2
  summary_after : 16
  samplerAct : ""
  rnn_channels : 30
  maskNoise : 0
  fgThreshold : 0.001
  betaCxt : 1
  checkpoint_after : 100
  betaCTL : 1
  realNoise : 0
  normScore : false
  actionY : 7
  latentSize : 16
  contextType : "max"
  nEpochs : 300
  noise : 0
  betaOrder : 1
  reset_control : false
  predictAngles : true
  weightDecay : 0.0001
  criticGT : false
  logRewardNormB : 0.01
  hyperFile : ""
  seq_length_test : 21
  matchNoise : 0
  ignoreMemThreshold : 0.5
  nThreads : 4
  betaCritic : 10
  gamma : 1
  orderby : "hung"
  criticNoise : 0
  learning_rate_decay : 2e-05
  rnn_layers : 1
  betaGradCTL : 1
  betaBCE : 1
  ignoreMem : false
  pyramidImage : false
  alpha : 0.99
  reset_context : false
  rewardVer : "match"
  critic_iter : 1
  crayonHost : "localhost"
  withVal : false
  reset_buffer : false
  epochNumber : 1
  dump : false
  use_pretrained : false
  validation_after : 10
  critic_warmup : 512
  pyramidFg : false
  ctl : ""
  betaCNN : 1
  criticTime : false
  optimState : "none"
}
---------------------------------	
--         Action: 7x7	
Setting the seed 128	
=> Creating model from file: models/preproc.lua	
Checking cache /home/Alexandrite/khoa/Segmentation_RL/CVPR/acis-master/code/gen/cvppp_train.t7	
Loading /home/Alexandrite/khoa/Segmentation_RL/CVPR/acis-master/code/gen/cvppp_train.t7	
Done

So I think the problem is placing the dataset A1.
I would like to clarify it again.
Is it correct that placing 128 images (plantxxx_label.png and plantxxx_rgb.png ) inside the path: acis-master\code\data\cvppp\ ?
Or I have to do something else?

Thank you again,
Khoa

arnike · 2019-10-31T11:22:06Z

Ah, sorry, I confused this with something else. Indeed, both the logs and checkpoints will be recorded in ./checkpoints directory.
The relative path to data is specified in opts.lua. There is usually a prefix A1_XYZ for the task, so the images with labels should be in:

acis-master/code/data/cvppp/A1_RAW/train
acis-master/code/data/cvppp/A1_RAW/val

for example.

Best,
Nikita

Khoa-NT · 2019-10-31T11:51:55Z

Hi, thank you for your quick reply.

I placed 128 images and 128 labels in

acis-master/code/data/cvppp/A1_RAW/train
acis-master/code/data/cvppp/A1_RAW/val

But everything still the same, nothing happens.

arnike · 2019-10-31T11:59:15Z

Do you see your GPUs being utilised, or does the process crush?

Khoa-NT · 2019-10-31T12:02:12Z

Yes, I see /home/khoa/torch/install/bin/luajit on all my GPUs.
I killed all the previous luajit tasks on GPU and run ./runs/cvppp_preproc.sh again but nothing happen.

arnike · 2019-10-31T12:03:53Z

OK, it seems then that it's training. Periodically, there should be lines printed into the log file with the status of training.

Best,
Nikita

Khoa-NT · 2019-10-31T12:12:08Z

Yes, but I still and only got the same 001_train.log

If there is nothing happen, then there will be error in the next step:
./runs/cvppp_preproc_save.sh
Because it requires the 300_preproc models but I don't have it from the first step.

Sincerely,
Khoa

Khoa-NT · 2019-11-03T06:52:24Z

hi @arnike,
Would you mind helping me by testing your code again?
I don't see any logs or saved model inside the checkpoint/

arnike · 2019-11-03T12:01:18Z

Hi @shaolinkhoa,
sorry for late replies. Did you change any default arguments by chance? How many GPUs are you using? cvppp_preproc.sh simply constructs the training command and executes in the background with nohup. You might just try executing it directly in the terminal and then breaking to see where (if) it hangs. If you see the GPUs are being utilised, my guess is that it just takes too long for some reason for printing the status update. You can control the interval between logging output with ``opt.summary_after'', try setting it to 1. Take a look at how it is used here.
Best,
Nikita

Khoa-NT · 2019-11-03T13:18:23Z

Hi @arnike ,
Glad to see your reply.

Did you change any default arguments by chance?

I can't install CUDA 8.0 and cuDNN-5.1 for Lua 5.2.4 so I follow this tut: installing LUA for CUDA 10.
That is the only difference between my project and yours. But I think it's not the problem.

How many GPUs are you using?

I'm using 8 GPUs. My GPUs status:

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0     13942      C   /home/khoa/torch/install/bin/luajit         1401MiB |
|    1     13942      C   /home/khoa/torch/install/bin/luajit          355MiB |
|    2     13942      C   /home/khoa/torch/install/bin/luajit          355MiB |
|    3     13942      C   /home/khoa/torch/install/bin/luajit          355MiB |
|    4     13942      C   /home/khoa/torch/install/bin/luajit          355MiB |
|    5     13942      C   /home/khoa/torch/install/bin/luajit          355MiB |
|    6     13942      C   /home/khoa/torch/install/bin/luajit          355MiB |
|    7     13942      C   /home/khoa/torch/install/bin/luajit          355MiB |
+-----------------------------------------------------------------------------+

cvppp_preproc.sh simply constructs the training command and executes in the background with nohup. You might just try executing it directly in the terminal and then breaking to see where (if) it hangs. If you see the GPUs are being utilised, my guess is that it just takes too long for some reason for printing the status update. You can control the interval between logging output with ``opt.summary_after'', try setting it to 1. Take a look at how it is used here.

I replace if iter % opt.summary_after == 0 then with if iter % 1 == 0 then.
After an hour, I still can't see any logs in checkpoints/cvppp_611_preproc/001_train.log and also nothing is printed on the terminal.

I'm not familiar with LUA but I think we need io.open in here , right ?

arnike · 2019-11-03T13:55:49Z

Hi,
I don't think io.open is a problem. Actually, can you try this on one GPU? I haven't tested this in a multi-GPU setting and the default options (e.g. batch size, iter size etc.) are currently set up for a single GPU only.
BTW, your GPU status shows memory allocation. Could you please show your full nvidia-smi, also with GPU utilisation?

Khoa-NT · 2019-11-03T14:30:35Z

Hi,
I just installed the LUA_TORCH 5.2.
Previously, I installed 5.1. ( My bad)

But I got this error:

Setting seed 130
/home/khoa/torch/install/bin/lua: /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:183: [thread 2 callback] ./datasets/cvppp.lua:138: bad argument #1 to 'size' (dimension 1 out of range of 0D tensor at /home/khoa/torch/pkg/torch/generic/Tensor.c:19)
stack traceback:
        [C]: in function 'size'
        ./datasets/cvppp.lua:138: in function <./datasets/cvppp.lua:137>
        (...tail calls...)
        [C]: in function 'xpcall'
        /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:234: in function 'callback'
        /home/khoa/torch/install/share/lua/5.2/threads/queue.lua:65: in function </home/khoa/torch/install/share/lua/5.2/threads/queue.lua:41>
        [C]: in function 'pcall'
        /home/khoa/torch/install/share/lua/5.2/threads/queue.lua:40: in function 'dojob'
        [string "  local Queue = require 'threads.queue'..."]:13: in main chunk
stack traceback:
        [C]: in function 'error'
        /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:183: in function 'dojob'
        /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:264: in function 'synchronize'
        /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:142: in function 'specific'
        /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:125: in function </home/khoa/torch/install/share/lua/5.2/threads/threads.lua:36>
        (...tail calls...)
        ./dataloader.lua:54: in function '__init'
        /home/khoa/torch/install/share/lua/5.2/torch/init.lua:91: in function </home/khoa/torch/install/share/lua/5.2/torch/init.lua:87>
        [C]: in function 'DataLoader'
        ./dataloader.lua:25: in function 'create'
        main_preproc.lua:71: in main chunk
        [C]: in function 'dofile'
        ...khoa/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
        [C]: in ?
Setting seed 131
Setting seed 132

@arnike Do you know this error ?

I don't know it's because of the dataset or not:
I placed all 128 kitti images ( plantxxx_rgb.png) and labels (plantxxx_label.png) in in acis-master\code\data\cvppp\A1_RAW\train and also in acis-master\code\data\cvppp\A1_RAW\val.

Ex: In acis-master/code/data/cvppp/A1_RAW/train/

khoa@topaz: acis-master/code$ ls data/cvppp/A1_RAW/train/
plant016_rgb.png    plant032_rgb.png    plant046_rgb.png    plant059_rgb.png    plant072_rgb.png    plant088_rgb.png    plant104_rgb.png    plant119_rgb.png    plant134_rgb.png    plant148_rgb.png
plant001_label.png  plant017_label.png  plant033_label.png  plant047_label.png  plant060_label.png  plant073_label.png  plant089_label.png  plant105_label.png  plant120_label.png  plant135_label.png  plant149_label.png
plant001_rgb.png    plant017_rgb.png    plant033_rgb.png    plant047_rgb.png    plant060_rgb.png    plant073_rgb.png    plant089_rgb.png    plant105_rgb.png    plant120_rgb.png    plant135_rgb.png    plant149_rgb.png
plant002_label.png  plant018_label.png  plant035_label.png  plant048_label.png  plant061_label.png  plant076_label.png  plant090_label.png  plant106_label.png  plant121_label.png  plant137_label.png  plant151_label.png
plant002_rgb.png    plant018_rgb.png    plant035_rgb.png    plant048_rgb.png    plant061_rgb.png    plant076_rgb.png    plant090_rgb.png    plant106_rgb.png    plant121_rgb.png    plant137_rgb.png    plant151_rgb.png
plant005_label.png  plant020_label.png  plant036_label.png  plant049_label.png  plant062_label.png  plant077_label.png  plant091_label.png  plant107_label.png  plant123_label.png  plant138_label.png  plant152_label.png
plant005_rgb.png    plant020_rgb.png    plant036_rgb.png    plant049_rgb.png    plant062_rgb.png    plant077_rgb.png    plant091_rgb.png    plant107_rgb.png    plant123_rgb.png    plant138_rgb.png    plant152_rgb.png
plant006_label.png  plant021_label.png  plant037_label.png  plant050_label.png  plant063_label.png  plant078_label.png  plant092_label.png  plant108_label.png  plant124_label.png  plant139_label.png  plant153_label.png
plant006_rgb.png    plant021_rgb.png    plant037_rgb.png    plant050_rgb.png    plant063_rgb.png    plant078_rgb.png    plant092_rgb.png    plant108_rgb.png    plant124_rgb.png    plant139_rgb.png    plant153_rgb.png
plant007_label.png  plant022_label.png  plant038_label.png  plant051_label.png  plant064_label.png  plant079_label.png  plant094_label.png  plant109_label.png  plant126_label.png  plant141_label.png  plant154_label.png
plant007_rgb.png    plant022_rgb.png    plant038_rgb.png    plant051_rgb.png    plant064_rgb.png    plant079_rgb.png    plant094_rgb.png    plant109_rgb.png    plant126_rgb.png    plant141_rgb.png    plant154_rgb.png
plant008_label.png  plant024_label.png  plant039_label.png  plant052_label.png  plant065_label.png  plant080_label.png  plant096_label.png  plant110_label.png  plant127_label.png  plant142_label.png  plant156_label.png
plant008_rgb.png    plant024_rgb.png    plant039_rgb.png    plant052_rgb.png    plant065_rgb.png    plant080_rgb.png    plant096_rgb.png    plant110_rgb.png    plant127_rgb.png    plant142_rgb.png    plant156_rgb.png
plant010_label.png  plant026_label.png  plant040_label.png  plant053_label.png  plant067_label.png  plant082_label.png  plant098_label.png  plant113_label.png  plant128_label.png  plant143_label.png  plant159_label.png
plant010_rgb.png    plant026_rgb.png    plant040_rgb.png    plant053_rgb.png    plant067_rgb.png    plant082_rgb.png    plant098_rgb.png    plant113_rgb.png    plant128_rgb.png    plant143_rgb.png    plant159_rgb.png
plant011_label.png  plant027_label.png  plant042_label.png  plant054_label.png  plant068_label.png  plant083_label.png  plant099_label.png  plant114_label.png  plant129_label.png  plant144_label.png  plant161_label.png
plant011_rgb.png    plant027_rgb.png    plant042_rgb.png    plant054_rgb.png    plant068_rgb.png    plant083_rgb.png    plant099_rgb.png    plant114_rgb.png    plant129_rgb.png    plant144_rgb.png    plant161_rgb.png
plant012_label.png  plant029_label.png  plant043_label.png  plant055_label.png  plant069_label.png  plant084_label.png  plant100_label.png  plant115_label.png  plant130_label.png  plant145_label.png
plant012_rgb.png    plant029_rgb.png    plant043_rgb.png    plant055_rgb.png    plant069_rgb.png    plant084_rgb.png    plant100_rgb.png    plant115_rgb.png    plant130_rgb.png    plant145_rgb.png
plant013_label.png  plant030_label.png  plant044_label.png  plant057_label.png  plant070_label.png  plant085_label.png  plant101_label.png  plant116_label.png  plant132_label.png  plant146_label.png
plant013_rgb.png    plant030_rgb.png    plant044_rgb.png    plant057_rgb.png    plant070_rgb.png    plant085_rgb.png    plant101_rgb.png    plant116_rgb.png    plant132_rgb.png    plant146_rgb.png
plant015_label.png  plant031_label.png  plant045_label.png  plant058_label.png  plant071_label.png  plant086_label.png  plant102_label.png  plant118_label.png  plant133_label.png  plant147_label.png
plant015_rgb.png    plant031_rgb.png    plant045_rgb.png    plant058_rgb.png    plant071_rgb.png    plant086_rgb.png    plant102_rgb.png    plant118_rgb.png    plant133_rgb.png    plant147_rgb.png
plant016_label.png  plant032_label.png  plant046_label.png  plant059_label.png  plant072_label.png  plant088_label.png  plant104_label.png  plant119_label.png  plant134_label.png  plant148_label.png

Do I setup the dataset correctly ?

Khoa-NT · 2019-11-09T12:38:00Z

Hi @arnike,
Would you mind supporting me this error ?

Best regards,
Khoa

arnike · 2019-11-09T14:48:03Z

Hi Khoa,

yes, the data layout looks correct. Did you try specifying the absolute datapath here?

Best,
Nikita

Khoa-NT · 2019-11-09T15:03:06Z

Hi @arnike ,
Good to see you back.
I placed the data follow that path but now I'm stuck with this error:

Setting seed 130
/home/khoa/torch/install/bin/lua: /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:183: [thread 2 callback] ./datasets/cvppp.lua:138: bad argument #1 to 'size' (dimension 1 out of range of 0D tensor at /home/khoa/torch/pkg/torch/generic/Tensor.c:19)
stack traceback:
        [C]: in function 'size'
        ./datasets/cvppp.lua:138: in function <./datasets/cvppp.lua:137>
        (...tail calls...)
        [C]: in function 'xpcall'
        /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:234: in function 'callback'
        /home/khoa/torch/install/share/lua/5.2/threads/queue.lua:65: in function </home/khoa/torch/install/share/lua/5.2/threads/queue.lua:41>
        [C]: in function 'pcall'
        /home/khoa/torch/install/share/lua/5.2/threads/queue.lua:40: in function 'dojob'
        [string "  local Queue = require 'threads.queue'..."]:13: in main chunk
stack traceback:
        [C]: in function 'error'
        /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:183: in function 'dojob'
        /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:264: in function 'synchronize'
        /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:142: in function 'specific'
        /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:125: in function </home/khoa/torch/install/share/lua/5.2/threads/threads.lua:36>
        (...tail calls...)
        ./dataloader.lua:54: in function '__init'
        /home/khoa/torch/install/share/lua/5.2/torch/init.lua:91: in function </home/khoa/torch/install/share/lua/5.2/torch/init.lua:87>
        [C]: in function 'DataLoader'
        ./dataloader.lua:25: in function 'create'
        main_preproc.lua:71: in main chunk
        [C]: in function 'dofile'
        ...khoa/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
        [C]: in ?
Setting seed 131
Setting seed 132

Do you know how to fix it ?
thank you,
Khoa

arnike · 2019-11-09T17:16:26Z

Hi Khoa,
I'm not sure why this happens, but it seems it doesn't find ImageInfo here. Do you see something like Loading cvppp-train.t7 in the output? Do these files get generated in a path specified by opt.gen?
Best,
Nikita

Khoa-NT · 2019-11-10T17:38:50Z

Hi @arnike ,
I found the problem:

cvppp_train.t7 was an error file so I deleted it.
error in string.gsub function because of the name acis-master contains -. I renamed it to acis_master

But now I have another problem:

---------------------------------
--         Action: 7x7
Setting the seed 128
=> Creating model from file: models/preproc.lua
Checking cache /home/Alexandrite/khoa/Segmentation_RL/CVPR/acis_master/code/gen/cvppp_train.t7
Loading /home/Alexandrite/khoa/Segmentation_RL/CVPR/acis_master/code/gen/cvppp_train.t7
Done
Setting seed 132
Setting seed 129
Setting seed 130
Setting seed 131
Checking cache /home/Alexandrite/khoa/Segmentation_RL/CVPR/acis_master/code/gen/cvppp_val.t7
Loading /home/Alexandrite/khoa/Segmentation_RL/CVPR/acis_master/code/gen/cvppp_val.t7
Done
Setting seed 129
Setting seed 132
Setting seed 131
Setting seed 130
Starting training...
/home/khoa/torch/install/bin/lua: /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:183: [thread 1 callback] /home/khoa/torch/install/share/lua/5.2/image/init.lua:735: bad argument #1 to 'scaleSimple' (image.scale: src not 2 or 3 dimensional)
stack traceback:
        [C]: in function 'scaleSimple'
        /home/khoa/torch/install/share/lua/5.2/image/init.lua:735: in function 'scale'
        ./datasets/transforms.lua:55: in function 'transform'
        ./datasets/transforms.lua:21: in function 'preprocess'
        ./dataloader.lua:121: in function <./dataloader.lua:98>
        (...tail calls...)
        [C]: in function 'xpcall'
        /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:234: in function 'callback'
        /home/khoa/torch/install/share/lua/5.2/threads/queue.lua:65: in function </home/khoa/torch/install/share/lua/5.2/threads/queue.lua:41>
        [C]: in function 'pcall'
        /home/khoa/torch/install/share/lua/5.2/threads/queue.lua:40: in function 'dojob'
        [string "  local Queue = require 'threads.queue'..."]:15: in main chunk
stack traceback:
        [C]: in function 'error'
        /home/khoa/torch/install/share/lua/5.2/threads/threads.lua:183: in function 'dojob'
        ./dataloader.lua:248: in function 'for iterator'
        main_preproc.lua:146: in main chunk
        [C]: in function 'dofile'
        ...khoa/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
        [C]: in ?

Do you know this error?

The images I use have size 500x530 and bit depth = 32
The labels I use have size 500x530 and bit depth = 4

Best regards,
Khoa

Khoa-NT · 2019-11-13T04:12:50Z

Hi @arnike
I'm still finding the problem.
Can you give me any suggestions?

Best regards,
Khoa

arnike · 2019-11-27T11:02:57Z

Hi Khoa,
there seems to be a file format problem with the label image. I wonder if that's changed since.
Could you, please, share one image-mask pair you're using?
Best,
Nikita

Khoa-NT · 2019-11-28T01:56:12Z

Hi @arnike ,

Thank you for replying.

I upload all image-mask pairs in this gg drive

arnike · 2019-12-02T13:53:07Z

Hi Khoa,
there was indeed a small issue with label image pre-processing in the dataloader, but it should be fixed now.
Best,
Nikita

Khoa-NT changed the title ~~Location to place the dataset A1~~ How to setup the dataset A1 for training Oct 31, 2019

arnike added a commit that referenced this issue Dec 2, 2019

fixing #3 for main_preproc

3ab05ba

arnike closed this as completed Dec 2, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to setup the dataset A1 for training #3

How to setup the dataset A1 for training #3

Khoa-NT commented Oct 29, 2019

Khoa-NT commented Oct 31, 2019 •

edited

Loading

arnike commented Oct 31, 2019 •

edited

Loading

Khoa-NT commented Oct 31, 2019 •

edited

Loading

arnike commented Oct 31, 2019

Khoa-NT commented Oct 31, 2019

arnike commented Oct 31, 2019

Khoa-NT commented Oct 31, 2019

arnike commented Oct 31, 2019

Khoa-NT commented Oct 31, 2019

Khoa-NT commented Nov 3, 2019

arnike commented Nov 3, 2019

Khoa-NT commented Nov 3, 2019 •

edited

Loading

arnike commented Nov 3, 2019

Khoa-NT commented Nov 3, 2019 •

edited

Loading

Khoa-NT commented Nov 9, 2019

arnike commented Nov 9, 2019

Khoa-NT commented Nov 9, 2019

arnike commented Nov 9, 2019

Khoa-NT commented Nov 10, 2019

Khoa-NT commented Nov 13, 2019

arnike commented Nov 27, 2019

Khoa-NT commented Nov 28, 2019

arnike commented Dec 2, 2019

How to setup the dataset A1 for training #3

How to setup the dataset A1 for training #3

Comments

Khoa-NT commented Oct 29, 2019

Khoa-NT commented Oct 31, 2019 • edited Loading

arnike commented Oct 31, 2019 • edited Loading

Khoa-NT commented Oct 31, 2019 • edited Loading

arnike commented Oct 31, 2019

Khoa-NT commented Oct 31, 2019

arnike commented Oct 31, 2019

Khoa-NT commented Oct 31, 2019

arnike commented Oct 31, 2019

Khoa-NT commented Oct 31, 2019

Khoa-NT commented Nov 3, 2019

arnike commented Nov 3, 2019

Khoa-NT commented Nov 3, 2019 • edited Loading

arnike commented Nov 3, 2019

Khoa-NT commented Nov 3, 2019 • edited Loading

Khoa-NT commented Nov 9, 2019

arnike commented Nov 9, 2019

Khoa-NT commented Nov 9, 2019

arnike commented Nov 9, 2019

Khoa-NT commented Nov 10, 2019

Khoa-NT commented Nov 13, 2019

arnike commented Nov 27, 2019

Khoa-NT commented Nov 28, 2019

arnike commented Dec 2, 2019

Khoa-NT commented Oct 31, 2019 •

edited

Loading

arnike commented Oct 31, 2019 •

edited

Loading

Khoa-NT commented Oct 31, 2019 •

edited

Loading

Khoa-NT commented Nov 3, 2019 •

edited

Loading

Khoa-NT commented Nov 3, 2019 •

edited

Loading