隨手記錄 training log.
from 某一個checkpoint 436, 開始訓練, 並且 freeze down conv, from 436 to 4362
unpickled total 8072 examples
Starting training from epoch 0/39...
Epoch: [ 0], Batch: [ 0/ 404] | Total Time: 2s
d_loss: 1.3869, g_loss: 50.3334, const_loss: 0.0013, l1_loss: 38.9031, fm_loss: 0.0186, perc_loss: 10.4865, edge: 0.2305
Checkpoint step 100 reached, but saving starts after step 200.
Epoch: [ 0], Batch: [ 100/ 404] | Total Time: 3m 20s
d_loss: 1.3868, g_loss: 50.6237, const_loss: 0.0016, l1_loss: 38.7336, fm_loss: 0.0214, perc_loss: 10.9386, edge: 0.2352
Checkpoint saved at step 200
Epoch: [ 0], Batch: [ 200/ 404] | Total Time: 6m 46s
d_loss: 1.3869, g_loss: 49.8461, const_loss: 0.0012, l1_loss: 37.6563, fm_loss: 0.0204, perc_loss: 11.2248, edge: 0.2500
Checkpoint saved at step 300
Epoch: [ 0], Batch: [ 300/ 404] | Total Time: 10m 5s
d_loss: 1.3868, g_loss: 49.8416, const_loss: 0.0012, l1_loss: 38.7271, fm_loss: 0.0201, perc_loss: 10.1790, edge: 0.2207
Checkpoint saved at step 400
Epoch: [ 0], Batch: [ 400/ 404] | Total Time: 13m 25s
d_loss: 1.3869, g_loss: 49.8627, const_loss: 0.0014, l1_loss: 38.8669, fm_loss: 0.0211, perc_loss: 10.0617, edge: 0.2182
--- End of Epoch 0 --- Time: 810.9s ---
LR Scheduler stepped. Current LR G: 0.000243, LR D: 0.000243
Epoch: [ 1], Batch: [ 0/ 404] | Total Time: 13m 32s
d_loss: 1.3868, g_loss: 51.2730, const_loss: 0.0015, l1_loss: 39.3884, fm_loss: 0.0193, perc_loss: 10.9523, edge: 0.2188
Checkpoint saved at step 500
Epoch: [ 1], Batch: [ 100/ 404] | Total Time: 16m 52s
d_loss: 1.3868, g_loss: 49.4949, const_loss: 0.0010, l1_loss: 39.5543, fm_loss: 0.0213, perc_loss: 9.0160, edge: 0.2090
Checkpoint saved at step 600
Epoch: [ 1], Batch: [ 200/ 404] | Total Time: 20m 13s
d_loss: 1.3868, g_loss: 49.1722, const_loss: 0.0013, l1_loss: 38.0335, fm_loss: 0.0201, perc_loss: 10.1957, edge: 0.2282
Checkpoint saved at step 700
Epoch: [ 1], Batch: [ 300/ 404] | Total Time: 23m 34s
d_loss: 1.3871, g_loss: 48.2774, const_loss: 0.0015, l1_loss: 37.4702, fm_loss: 0.0185, perc_loss: 9.8669, edge: 0.2270
Checkpoint saved at step 800
Epoch: [ 1], Batch: [ 400/ 404] | Total Time: 26m 54s
d_loss: 1.3868, g_loss: 48.8425, const_loss: 0.0013, l1_loss: 37.7449, fm_loss: 0.0197, perc_loss: 10.1551, edge: 0.2281
--- End of Epoch 1 --- Time: 808.3s ---
LR Scheduler stepped. Current LR G: 0.000242, LR D: 0.000242
Epoch: [ 2], Batch: [ 0/ 404] | Total Time: 27m 1s
d_loss: 1.3868, g_loss: 48.7556, const_loss: 0.0011, l1_loss: 38.1520, fm_loss: 0.0188, perc_loss: 9.6703, edge: 0.2201
Checkpoint saved at step 900
Epoch: [ 2], Batch: [ 100/ 404] | Total Time: 30m 21s
d_loss: 1.3867, g_loss: 47.4981, const_loss: 0.0013, l1_loss: 37.2582, fm_loss: 0.0182, perc_loss: 9.3143, edge: 0.2127
Checkpoint saved at step 1000
Epoch: [ 2], Batch: [ 200/ 404] | Total Time: 33m 40s
d_loss: 1.3868, g_loss: 48.6449, const_loss: 0.0012, l1_loss: 38.0945, fm_loss: 0.0190, perc_loss: 9.6087, edge: 0.2282
Checkpoint saved at step 1100
Epoch: [ 2], Batch: [ 300/ 404] | Total Time: 37m 0s
d_loss: 1.3871, g_loss: 49.0583, const_loss: 0.0013, l1_loss: 39.0706, fm_loss: 0.0180, perc_loss: 9.0667, edge: 0.2083
Checkpoint saved at step 1200
Epoch: [ 2], Batch: [ 400/ 404] | Total Time: 40m 20s
d_loss: 1.3868, g_loss: 49.3098, const_loss: 0.0011, l1_loss: 38.6744, fm_loss: 0.0181, perc_loss: 9.6955, edge: 0.2274
--- End of Epoch 2 --- Time: 806.2s ---
LR Scheduler stepped. Current LR G: 0.000240, LR D: 0.000240
Epoch: [ 3], Batch: [ 0/ 404] | Total Time: 40m 27s
d_loss: 1.3867, g_loss: 48.4190, const_loss: 0.0012, l1_loss: 37.6385, fm_loss: 0.0167, perc_loss: 9.8418, edge: 0.2275
Checkpoint saved at step 1300
Epoch: [ 3], Batch: [ 100/ 404] | Total Time: 43m 46s
d_loss: 1.3878, g_loss: 48.2685, const_loss: 0.0009, l1_loss: 38.1845, fm_loss: 0.0175, perc_loss: 9.1591, edge: 0.2131
Checkpoint saved at step 1400
Epoch: [ 3], Batch: [ 200/ 404] | Total Time: 47m 6s
d_loss: 1.3867, g_loss: 48.2700, const_loss: 0.0010, l1_loss: 38.7207, fm_loss: 0.0168, perc_loss: 8.6360, edge: 0.2022
Checkpoint saved at step 1500
Epoch: [ 3], Batch: [ 300/ 404] | Total Time: 50m 25s
d_loss: 1.3870, g_loss: 48.3382, const_loss: 0.0012, l1_loss: 37.6600, fm_loss: 0.0183, perc_loss: 9.7520, edge: 0.2134
Checkpoint saved at step 1600
Epoch: [ 3], Batch: [ 400/ 404] | Total Time: 53m 45s
d_loss: 1.3870, g_loss: 48.6295, const_loss: 0.0011, l1_loss: 39.3946, fm_loss: 0.0153, perc_loss: 8.3330, edge: 0.1926
--- End of Epoch 3 --- Time: 805.5s ---
LR Scheduler stepped. Current LR G: 0.000237, LR D: 0.000237
Epoch: [ 4], Batch: [ 0/ 404] | Total Time: 53m 52s
d_loss: 1.3889, g_loss: 47.3585, const_loss: 0.0010, l1_loss: 37.1291, fm_loss: 0.0175, perc_loss: 9.2912, edge: 0.2269
Checkpoint saved at step 1700
Epoch: [ 4], Batch: [ 100/ 404] | Total Time: 57m 12s
d_loss: 1.3867, g_loss: 47.8745, const_loss: 0.0009, l1_loss: 38.1769, fm_loss: 0.0171, perc_loss: 8.7823, edge: 0.2038
Checkpoint saved at step 1800
Epoch: [ 4], Batch: [ 200/ 404] | Total Time: 1h 31s
d_loss: 1.3867, g_loss: 47.2119, const_loss: 0.0013, l1_loss: 37.5022, fm_loss: 0.0178, perc_loss: 8.7953, edge: 0.2020
Checkpoint saved at step 1900
Epoch: [ 4], Batch: [ 300/ 404] | Total Time: 1h 3m 51s
d_loss: 1.3871, g_loss: 46.6843, const_loss: 0.0011, l1_loss: 36.0923, fm_loss: 0.0154, perc_loss: 9.6592, edge: 0.2235
Checkpoint saved at step 2000
Epoch: [ 4], Batch: [ 400/ 404] | Total Time: 1h 7m 10s
d_loss: 1.3869, g_loss: 47.7873, const_loss: 0.0009, l1_loss: 37.4505, fm_loss: 0.0186, perc_loss: 9.4093, edge: 0.2151
--- End of Epoch 4 --- Time: 804.9s ---
LR Scheduler stepped. Current LR G: 0.000234, LR D: 0.000234
Epoch: [ 5], Batch: [ 0/ 404] | Total Time: 1h 7m 17s
d_loss: 1.3871, g_loss: 48.7753, const_loss: 0.0010, l1_loss: 39.3204, fm_loss: 0.0213, perc_loss: 8.5372, edge: 0.2020
Checkpoint saved at step 2100
Epoch: [ 5], Batch: [ 100/ 404] | Total Time: 1h 10m 36s
d_loss: 1.3868, g_loss: 46.6811, const_loss: 0.0010, l1_loss: 35.9192, fm_loss: 0.0156, perc_loss: 9.8247, edge: 0.2272
Checkpoint saved at step 2200
Epoch: [ 5], Batch: [ 200/ 404] | Total Time: 1h 13m 56s
d_loss: 1.3871, g_loss: 46.8245, const_loss: 0.0009, l1_loss: 37.1959, fm_loss: 0.0172, perc_loss: 8.7044, edge: 0.2128
Checkpoint saved at step 2300
Epoch: [ 5], Batch: [ 300/ 404] | Total Time: 1h 17m 16s
d_loss: 1.3869, g_loss: 45.8490, const_loss: 0.0010, l1_loss: 36.4918, fm_loss: 0.0143, perc_loss: 8.4467, edge: 0.2024
Checkpoint saved at step 2400
Epoch: [ 5], Batch: [ 400/ 404] | Total Time: 1h 20m 36s
d_loss: 1.3872, g_loss: 48.7075, const_loss: 0.0013, l1_loss: 39.8774, fm_loss: 0.0150, perc_loss: 7.9354, edge: 0.1851
--- End of Epoch 5 --- Time: 806.3s ---
LR Scheduler stepped. Current LR G: 0.000230, LR D: 0.000230
Epoch: [ 6], Batch: [ 0/ 404] | Total Time: 1h 20m 44s
d_loss: 1.3870, g_loss: 48.1324, const_loss: 0.0011, l1_loss: 38.1683, fm_loss: 0.0177, perc_loss: 9.0456, edge: 0.2064
Checkpoint saved at step 2500
Epoch: [ 6], Batch: [ 100/ 404] | Total Time: 1h 24m 3s
d_loss: 1.3868, g_loss: 47.9302, const_loss: 0.0011, l1_loss: 38.1133, fm_loss: 0.0165, perc_loss: 8.9057, edge: 0.2002
Checkpoint saved at step 2600
Epoch: [ 6], Batch: [ 200/ 404] | Total Time: 1h 27m 23s
d_loss: 1.3870, g_loss: 46.2337, const_loss: 0.0011, l1_loss: 36.9181, fm_loss: 0.0147, perc_loss: 8.4065, edge: 0.2004
Checkpoint saved at step 2700
Epoch: [ 6], Batch: [ 300/ 404] | Total Time: 1h 30m 42s
d_loss: 1.3872, g_loss: 45.8828, const_loss: 0.0009, l1_loss: 36.5023, fm_loss: 0.0142, perc_loss: 8.4680, edge: 0.2044
Checkpoint saved at step 2800
Epoch: [ 6], Batch: [ 400/ 404] | Total Time: 1h 34m 2s
d_loss: 1.3869, g_loss: 48.5134, const_loss: 0.0010, l1_loss: 39.9758, fm_loss: 0.0157, perc_loss: 7.6407, edge: 0.1867
--- End of Epoch 6 --- Time: 805.3s ---
LR Scheduler stepped. Current LR G: 0.000225, LR D: 0.000225
Epoch: [ 7], Batch: [ 0/ 404] | Total Time: 1h 34m 9s
d_loss: 1.3868, g_loss: 46.1921, const_loss: 0.0009, l1_loss: 36.9415, fm_loss: 0.0133, perc_loss: 8.3454, edge: 0.1981
Checkpoint saved at step 2900
Epoch: [ 7], Batch: [ 100/ 404] | Total Time: 1h 37m 28s
d_loss: 1.3871, g_loss: 48.7168, const_loss: 0.0009, l1_loss: 39.7169, fm_loss: 0.0160, perc_loss: 8.0980, edge: 0.1917
Checkpoint saved at step 3000
Epoch: [ 7], Batch: [ 200/ 404] | Total Time: 1h 40m 48s
d_loss: 1.3867, g_loss: 47.8504, const_loss: 0.0011, l1_loss: 38.3773, fm_loss: 0.0152, perc_loss: 8.5623, edge: 0.2016
Checkpoint saved at step 3100
Epoch: [ 7], Batch: [ 300/ 404] | Total Time: 1h 44m 7s
d_loss: 1.3871, g_loss: 48.2163, const_loss: 0.0012, l1_loss: 39.4610, fm_loss: 0.0163, perc_loss: 7.8626, edge: 0.1819
Checkpoint saved at step 3200
Epoch: [ 7], Batch: [ 400/ 404] | Total Time: 1h 47m 26s
d_loss: 1.3867, g_loss: 46.7838, const_loss: 0.0008, l1_loss: 37.4049, fm_loss: 0.0152, perc_loss: 8.4674, edge: 0.2026
--- End of Epoch 7 --- Time: 804.5s ---
LR Scheduler stepped. Current LR G: 0.000220, LR D: 0.000220
Epoch: [ 8], Batch: [ 0/ 404] | Total Time: 1h 47m 33s
d_loss: 1.3868, g_loss: 47.7452, const_loss: 0.0010, l1_loss: 39.0489, fm_loss: 0.0159, perc_loss: 7.7983, edge: 0.1879
Checkpoint saved at step 3300
Epoch: [ 8], Batch: [ 100/ 404] | Total Time: 1h 50m 53s
d_loss: 1.3881, g_loss: 45.4044, const_loss: 0.0008, l1_loss: 36.4637, fm_loss: 0.0130, perc_loss: 8.0424, edge: 0.1918
Checkpoint saved at step 3400
Epoch: [ 8], Batch: [ 200/ 404] | Total Time: 1h 54m 18s
d_loss: 1.3869, g_loss: 46.0476, const_loss: 0.0006, l1_loss: 36.6350, fm_loss: 0.0128, perc_loss: 8.5132, edge: 0.1932
Checkpoint saved at step 3500
Epoch: [ 8], Batch: [ 300/ 404] | Total Time: 1h 57m 37s
d_loss: 1.3867, g_loss: 44.8839, const_loss: 0.0007, l1_loss: 35.8059, fm_loss: 0.0125, perc_loss: 8.1718, edge: 0.2001
Checkpoint saved at step 3600
Epoch: [ 8], Batch: [ 400/ 404] | Total Time: 2h 59s
d_loss: 1.3868, g_loss: 46.2760, const_loss: 0.0009, l1_loss: 36.8915, fm_loss: 0.0139, perc_loss: 8.4797, edge: 0.1972
--- End of Epoch 8 --- Time: 812.9s ---
LR Scheduler stepped. Current LR G: 0.000214, LR D: 0.000214
Epoch: [ 9], Batch: [ 0/ 404] | Total Time: 2h 1m 6s
d_loss: 1.3868, g_loss: 48.0222, const_loss: 0.0009, l1_loss: 39.0547, fm_loss: 0.0157, perc_loss: 8.0652, edge: 0.1924
Checkpoint saved at step 3700
Epoch: [ 9], Batch: [ 100/ 404] | Total Time: 2h 4m 31s
d_loss: 1.3868, g_loss: 46.6141, const_loss: 0.0009, l1_loss: 38.3572, fm_loss: 0.0134, perc_loss: 7.3772, edge: 0.1725
Checkpoint saved at step 3800
Epoch: [ 9], Batch: [ 200/ 404] | Total Time: 2h 7m 55s
d_loss: 1.3867, g_loss: 47.2245, const_loss: 0.0008, l1_loss: 38.1773, fm_loss: 0.0140, perc_loss: 8.1396, edge: 0.1994
Checkpoint saved at step 3900
Epoch: [ 9], Batch: [ 300/ 404] | Total Time: 2h 11m 20s
d_loss: 1.3868, g_loss: 46.6189, const_loss: 0.0012, l1_loss: 38.0776, fm_loss: 0.0143, perc_loss: 7.6462, edge: 0.1868
Checkpoint saved at step 4000
Epoch: [ 9], Batch: [ 400/ 404] | Total Time: 2h 14m 45s
d_loss: 1.3868, g_loss: 46.2391, const_loss: 0.0009, l1_loss: 36.9850, fm_loss: 0.0148, perc_loss: 8.3422, edge: 0.2029
--- End of Epoch 9 --- Time: 826.0s ---
LR Scheduler stepped. Current LR G: 0.000208, LR D: 0.000208
Epoch: [ 10], Batch: [ 0/ 404] | Total Time: 2h 14m 52s
d_loss: 1.3869, g_loss: 45.5681, const_loss: 0.0008, l1_loss: 36.8380, fm_loss: 0.0130, perc_loss: 7.8444, edge: 0.1792
Checkpoint saved at step 4100
Epoch: [ 10], Batch: [ 100/ 404] | Total Time: 2h 18m 20s
d_loss: 1.3869, g_loss: 45.5430, const_loss: 0.0010, l1_loss: 36.8723, fm_loss: 0.0127, perc_loss: 7.7820, edge: 0.1817
Checkpoint saved at step 4200
Epoch: [ 10], Batch: [ 200/ 404] | Total Time: 2h 21m 43s
d_loss: 1.3868, g_loss: 46.0187, const_loss: 0.0008, l1_loss: 37.3930, fm_loss: 0.0126, perc_loss: 7.7288, edge: 0.1906
Checkpoint saved at step 4300
Epoch: [ 10], Batch: [ 300/ 404] | Total Time: 2h 25m 3s
d_loss: 1.3871, g_loss: 45.3588, const_loss: 0.0009, l1_loss: 36.4508, fm_loss: 0.0134, perc_loss: 8.0068, edge: 0.1940
Checkpoint saved at step 4400
Epoch: [ 10], Batch: [ 400/ 404] | Total Time: 2h 28m 22s
d_loss: 1.3868, g_loss: 46.7172, const_loss: 0.0008, l1_loss: 38.2640, fm_loss: 0.0132, perc_loss: 7.5711, edge: 0.1747
--- End of Epoch 10 --- Time: 816.6s ---
LR Scheduler stepped. Current LR G: 0.000201, LR D: 0.000201
Epoch: [ 11], Batch: [ 0/ 404] | Total Time: 2h 28m 29s
d_loss: 1.3869, g_loss: 45.7797, const_loss: 0.0010, l1_loss: 37.3163, fm_loss: 0.0126, perc_loss: 7.5723, edge: 0.1846
Checkpoint saved at step 4500
Epoch: [ 11], Batch: [ 100/ 404] | Total Time: 2h 31m 49s
d_loss: 1.3868, g_loss: 44.4696, const_loss: 0.0007, l1_loss: 35.4980, fm_loss: 0.0120, perc_loss: 8.0765, edge: 0.1895
Checkpoint saved at step 4600
Epoch: [ 11], Batch: [ 200/ 404] | Total Time: 2h 35m 8s
d_loss: 1.3868, g_loss: 46.0298, const_loss: 0.0008, l1_loss: 37.2192, fm_loss: 0.0136, perc_loss: 7.9135, edge: 0.1893
Checkpoint saved at step 4700
Epoch: [ 11], Batch: [ 300/ 404] | Total Time: 2h 38m 33s
d_loss: 1.3870, g_loss: 47.4978, const_loss: 0.0007, l1_loss: 38.5342, fm_loss: 0.0157, perc_loss: 8.0639, edge: 0.1900
Checkpoint saved at step 4800
Epoch: [ 11], Batch: [ 400/ 404] | Total Time: 2h 41m 52s
d_loss: 1.3868, g_loss: 46.0109, const_loss: 0.0008, l1_loss: 37.1338, fm_loss: 0.0131, perc_loss: 7.9922, edge: 0.1777
--- End of Epoch 11 --- Time: 810.2s ---
LR Scheduler stepped. Current LR G: 0.000193, LR D: 0.000193
Epoch: [ 12], Batch: [ 0/ 404] | Total Time: 2h 41m 59s
d_loss: 1.3873, g_loss: 46.3447, const_loss: 0.0010, l1_loss: 38.1679, fm_loss: 0.0121, perc_loss: 7.2971, edge: 0.1733
Checkpoint saved at step 4900
Epoch: [ 12], Batch: [ 100/ 404] | Total Time: 2h 45m 24s
d_loss: 1.3869, g_loss: 45.8750, const_loss: 0.0009, l1_loss: 36.7592, fm_loss: 0.0130, perc_loss: 8.2151, edge: 0.1934
Checkpoint saved at step 5000
Epoch: [ 12], Batch: [ 200/ 404] | Total Time: 2h 48m 49s
d_loss: 1.3868, g_loss: 46.2548, const_loss: 0.0008, l1_loss: 38.3599, fm_loss: 0.0126, perc_loss: 7.0206, edge: 0.1675
Checkpoint saved at step 5100
Epoch: [ 12], Batch: [ 300/ 404] | Total Time: 2h 52m 9s
d_loss: 1.3867, g_loss: 43.7772, const_loss: 0.0007, l1_loss: 35.0042, fm_loss: 0.0116, perc_loss: 7.8832, edge: 0.1841
Checkpoint saved at step 5200
Epoch: [ 12], Batch: [ 400/ 404] | Total Time: 2h 55m 33s
d_loss: 1.3867, g_loss: 46.5131, const_loss: 0.0008, l1_loss: 38.4010, fm_loss: 0.0124, perc_loss: 7.2480, edge: 0.1576
--- End of Epoch 12 --- Time: 821.0s ---
LR Scheduler stepped. Current LR G: 0.000185, LR D: 0.000185
Epoch: [ 13], Batch: [ 0/ 404] | Total Time: 2h 55m 40s
d_loss: 1.3867, g_loss: 46.2999, const_loss: 0.0008, l1_loss: 37.9654, fm_loss: 0.0109, perc_loss: 7.4599, edge: 0.1696
Checkpoint saved at step 5300
Epoch: [ 13], Batch: [ 100/ 404] | Total Time: 2h 58m 59s
d_loss: 1.3869, g_loss: 46.0993, const_loss: 0.0008, l1_loss: 37.3394, fm_loss: 0.0125, perc_loss: 7.8749, edge: 0.1784
Checkpoint saved at step 5400
Epoch: [ 13], Batch: [ 200/ 404] | Total Time: 3h 2m 26s
d_loss: 1.3868, g_loss: 45.2989, const_loss: 0.0008, l1_loss: 36.8141, fm_loss: 0.0120, perc_loss: 7.6048, edge: 0.1744
Checkpoint saved at step 5500
Epoch: [ 13], Batch: [ 300/ 404] | Total Time: 3h 5m 49s
d_loss: 1.3869, g_loss: 46.1642, const_loss: 0.0008, l1_loss: 37.9691, fm_loss: 0.0121, perc_loss: 7.3074, edge: 0.1813
Checkpoint saved at step 5600
Epoch: [ 13], Batch: [ 400/ 404] | Total Time: 3h 9m 14s
d_loss: 1.3869, g_loss: 47.8757, const_loss: 0.0010, l1_loss: 40.1760, fm_loss: 0.0137, perc_loss: 6.8358, edge: 0.1559
--- End of Epoch 13 --- Time: 820.7s ---
LR Scheduler stepped. Current LR G: 0.000177, LR D: 0.000177
Epoch: [ 14], Batch: [ 0/ 404] | Total Time: 3h 9m 21s
d_loss: 1.3877, g_loss: 45.6725, const_loss: 0.0006, l1_loss: 37.2671, fm_loss: 0.0116, perc_loss: 7.5177, edge: 0.1822
Checkpoint saved at step 5700
Epoch: [ 14], Batch: [ 100/ 404] | Total Time: 3h 12m 43s
d_loss: 1.3867, g_loss: 44.8660, const_loss: 0.0008, l1_loss: 36.4035, fm_loss: 0.0109, perc_loss: 7.5707, edge: 0.1868
Checkpoint saved at step 5800
Epoch: [ 14], Batch: [ 200/ 404] | Total Time: 3h 16m 7s
d_loss: 1.3868, g_loss: 46.2573, const_loss: 0.0008, l1_loss: 38.0839, fm_loss: 0.0117, perc_loss: 7.3044, edge: 0.1632
Checkpoint saved at step 5900
Epoch: [ 14], Batch: [ 300/ 404] | Total Time: 3h 19m 31s
d_loss: 1.3870, g_loss: 44.9992, const_loss: 0.0007, l1_loss: 37.0555, fm_loss: 0.0105, perc_loss: 7.0766, edge: 0.1626
Checkpoint saved at step 6000
Epoch: [ 14], Batch: [ 400/ 404] | Total Time: 3h 22m 57s
d_loss: 1.3868, g_loss: 45.9360, const_loss: 0.0007, l1_loss: 37.8453, fm_loss: 0.0111, perc_loss: 7.2132, edge: 0.1723
--- End of Epoch 14 --- Time: 822.8s ---
LR Scheduler stepped. Current LR G: 0.000168, LR D: 0.000168
Epoch: [ 15], Batch: [ 0/ 404] | Total Time: 3h 23m 4s
d_loss: 1.3868, g_loss: 44.3930, const_loss: 0.0008, l1_loss: 36.3449, fm_loss: 0.0102, perc_loss: 7.1670, edge: 0.1767
Checkpoint saved at step 6100
Epoch: [ 15], Batch: [ 100/ 404] | Total Time: 3h 26m 27s
d_loss: 1.3869, g_loss: 46.4168, const_loss: 0.0008, l1_loss: 38.2578, fm_loss: 0.0146, perc_loss: 7.2751, edge: 0.1753
Checkpoint saved at step 6200
Epoch: [ 15], Batch: [ 200/ 404] | Total Time: 3h 29m 51s
d_loss: 1.3869, g_loss: 44.8256, const_loss: 0.0007, l1_loss: 36.4796, fm_loss: 0.0105, perc_loss: 7.4557, edge: 0.1856
Checkpoint saved at step 6300
Epoch: [ 15], Batch: [ 300/ 404] | Total Time: 3h 33m 12s
d_loss: 1.3868, g_loss: 45.7407, const_loss: 0.0009, l1_loss: 37.8039, fm_loss: 0.0118, perc_loss: 7.0631, edge: 0.1677
from 4364 to 4366
Model 4364 loaded successfully
unpickled total 8072 examples
Starting training from epoch 0/39...
Epoch: [ 0], Batch: [ 0/ 404] | Total Time: 4s
d_loss: 1.3868, g_loss: 45.0375, const_loss: 0.0009, l1_loss: 36.6432, fm_loss: 0.0109, perc_loss: 7.5201, edge: 0.1692
Checkpoint step 100 reached, but saving starts after step 200.
Epoch: [ 0], Batch: [ 100/ 404] | Total Time: 3m 40s
d_loss: 1.3868, g_loss: 49.5123, const_loss: 0.0035, l1_loss: 39.4749, fm_loss: 0.0120, perc_loss: 9.1204, edge: 0.2082
Epoch: [ 0], Batch: [ 200/ 404] | Total Time: 7m 26s
d_loss: 1.3868, g_loss: 47.4852, const_loss: 0.0021, l1_loss: 37.5089, fm_loss: 0.0125, perc_loss: 9.0436, edge: 0.2247
Epoch: [ 0], Batch: [ 300/ 404] | Total Time: 11m 7s
d_loss: 1.3870, g_loss: 48.7978, const_loss: 0.0024, l1_loss: 38.6334, fm_loss: 0.0128, perc_loss: 9.2379, edge: 0.2180
Epoch: [ 0], Batch: [ 400/ 404] | Total Time: 14m 47s
d_loss: 1.3868, g_loss: 48.3058, const_loss: 0.0025, l1_loss: 38.5833, fm_loss: 0.0119, perc_loss: 8.8152, edge: 0.1996
--- End of Epoch 0 --- Time: 893.6s ---
LR Scheduler stepped. Current LR G: 0.000235, LR D: 0.000235
Epoch: [ 1], Batch: [ 0/ 404] | Total Time: 14m 55s
d_loss: 1.3868, g_loss: 48.2755, const_loss: 0.0031, l1_loss: 38.5546, fm_loss: 0.0129, perc_loss: 8.7984, edge: 0.2131
Epoch: [ 1], Batch: [ 100/ 404] | Total Time: 18m 36s
d_loss: 1.3868, g_loss: 47.1589, const_loss: 0.0031, l1_loss: 37.4362, fm_loss: 0.0127, perc_loss: 8.8115, edge: 0.2021
Epoch: [ 1], Batch: [ 200/ 404] | Total Time: 22m 17s
d_loss: 1.3869, g_loss: 48.9361, const_loss: 0.0020, l1_loss: 40.0920, fm_loss: 0.0154, perc_loss: 7.9551, edge: 0.1782
Epoch: [ 1], Batch: [ 300/ 404] | Total Time: 25m 57s
d_loss: 1.3867, g_loss: 46.3560, const_loss: 0.0029, l1_loss: 36.7961, fm_loss: 0.0117, perc_loss: 8.6452, edge: 0.2067
Epoch: [ 1], Batch: [ 400/ 404] | Total Time: 29m 37s
d_loss: 1.3869, g_loss: 46.6964, const_loss: 0.0027, l1_loss: 37.8695, fm_loss: 0.0112, perc_loss: 7.9268, edge: 0.1929
--- End of Epoch 1 --- Time: 890.0s ---
LR Scheduler stepped. Current LR G: 0.000234, LR D: 0.000234
Epoch: [ 2], Batch: [ 0/ 404] | Total Time: 29m 45s
d_loss: 1.3867, g_loss: 46.8502, const_loss: 0.0021, l1_loss: 38.6274, fm_loss: 0.0118, perc_loss: 7.3363, edge: 0.1793
Epoch: [ 2], Batch: [ 100/ 404] | Total Time: 33m 26s
d_loss: 1.3870, g_loss: 47.0021, const_loss: 0.0023, l1_loss: 37.3924, fm_loss: 0.0112, perc_loss: 8.6967, edge: 0.2062
Epoch: [ 2], Batch: [ 200/ 404] | Total Time: 37m 6s
d_loss: 1.3870, g_loss: 48.1564, const_loss: 0.0024, l1_loss: 39.3354, fm_loss: 0.0109, perc_loss: 7.9289, edge: 0.1855
Epoch: [ 2], Batch: [ 300/ 404] | Total Time: 40m 47s
d_loss: 1.3872, g_loss: 46.5216, const_loss: 0.0022, l1_loss: 36.9838, fm_loss: 0.0105, perc_loss: 8.6317, edge: 0.2001
Epoch: [ 2], Batch: [ 400/ 404] | Total Time: 44m 27s
d_loss: 1.3869, g_loss: 46.4278, const_loss: 0.0018, l1_loss: 36.7253, fm_loss: 0.0103, perc_loss: 8.7810, edge: 0.2161
--- End of Epoch 2 --- Time: 890.1s ---
LR Scheduler stepped. Current LR G: 0.000232, LR D: 0.000232
Epoch: [ 3], Batch: [ 0/ 404] | Total Time: 44m 35s
d_loss: 1.3878, g_loss: 47.3694, const_loss: 0.0029, l1_loss: 39.0800, fm_loss: 0.0101, perc_loss: 7.3980, edge: 0.1849
Epoch: [ 3], Batch: [ 100/ 404] | Total Time: 48m 16s
d_loss: 1.3867, g_loss: 47.4595, const_loss: 0.0028, l1_loss: 39.0318, fm_loss: 0.0117, perc_loss: 7.5382, edge: 0.1817
Epoch: [ 3], Batch: [ 200/ 404] | Total Time: 51m 56s
d_loss: 1.3870, g_loss: 45.5972, const_loss: 0.0023, l1_loss: 36.8670, fm_loss: 0.0097, perc_loss: 7.8368, edge: 0.1881
Epoch: [ 3], Batch: [ 300/ 404] | Total Time: 55m 36s
d_loss: 1.3867, g_loss: 47.9765, const_loss: 0.0024, l1_loss: 38.3661, fm_loss: 0.0124, perc_loss: 8.6836, edge: 0.2186
Epoch: [ 3], Batch: [ 400/ 404] | Total Time: 59m 17s
d_loss: 1.3867, g_loss: 46.1025, const_loss: 0.0025, l1_loss: 37.1707, fm_loss: 0.0113, perc_loss: 8.0240, edge: 0.2005
--- End of Epoch 3 --- Time: 889.6s ---
LR Scheduler stepped. Current LR G: 0.000229, LR D: 0.000229
Epoch: [ 4], Batch: [ 0/ 404] | Total Time: 59m 25s
d_loss: 1.3867, g_loss: 45.4111, const_loss: 0.0026, l1_loss: 36.6954, fm_loss: 0.0102, perc_loss: 7.8188, edge: 0.1906
Epoch: [ 4], Batch: [ 100/ 404] | Total Time: 1h 3m 5s
d_loss: 1.3868, g_loss: 46.8678, const_loss: 0.0020, l1_loss: 38.1686, fm_loss: 0.0113, perc_loss: 7.8036, edge: 0.1890
Epoch: [ 4], Batch: [ 200/ 404] | Total Time: 1h 6m 46s
d_loss: 1.3868, g_loss: 46.7259, const_loss: 0.0021, l1_loss: 37.7158, fm_loss: 0.0107, perc_loss: 8.0987, edge: 0.2058
Epoch: [ 4], Batch: [ 300/ 404] | Total Time: 1h 10m 27s
d_loss: 1.3870, g_loss: 47.4380, const_loss: 0.0024, l1_loss: 38.6533, fm_loss: 0.0105, perc_loss: 7.8996, edge: 0.1789
Epoch: [ 4], Batch: [ 400/ 404] | Total Time: 1h 14m 7s
d_loss: 1.3869, g_loss: 45.8967, const_loss: 0.0023, l1_loss: 37.3110, fm_loss: 0.0119, perc_loss: 7.6840, edge: 0.1942
--- End of Epoch 4 --- Time: 890.1s ---
LR Scheduler stepped. Current LR G: 0.000226, LR D: 0.000226
Epoch: [ 5], Batch: [ 0/ 404] | Total Time: 1h 14m 15s
d_loss: 1.3873, g_loss: 46.1631, const_loss: 0.0019, l1_loss: 37.0132, fm_loss: 0.0111, perc_loss: 8.2485, edge: 0.1949
Epoch: [ 5], Batch: [ 100/ 404] | Total Time: 1h 17m 59s
d_loss: 1.3868, g_loss: 48.3828, const_loss: 0.0022, l1_loss: 40.2578, fm_loss: 0.0115, perc_loss: 7.2423, edge: 0.1757
Epoch: [ 5], Batch: [ 200/ 404] | Total Time: 1h 21m 39s
d_loss: 1.3869, g_loss: 46.8092, const_loss: 0.0018, l1_loss: 38.4948, fm_loss: 0.0105, perc_loss: 7.4239, edge: 0.1848
Epoch: [ 5], Batch: [ 300/ 404] | Total Time: 1h 25m 20s
d_loss: 1.3869, g_loss: 47.5913, const_loss: 0.0021, l1_loss: 39.0686, fm_loss: 0.0106, perc_loss: 7.6358, edge: 0.1809
Epoch: [ 5], Batch: [ 400/ 404] | Total Time: 1h 29m 1s
d_loss: 1.3867, g_loss: 45.8116, const_loss: 0.0017, l1_loss: 36.8833, fm_loss: 0.0101, perc_loss: 8.0345, edge: 0.1887
--- End of Epoch 5 --- Time: 893.6s ---
LR Scheduler stepped. Current LR G: 0.000222, LR D: 0.000222
Epoch: [ 6], Batch: [ 0/ 404] | Total Time: 1h 29m 9s
d_loss: 1.3870, g_loss: 47.0662, const_loss: 0.0021, l1_loss: 38.7755, fm_loss: 0.0116, perc_loss: 7.4164, edge: 0.1673
Epoch: [ 6], Batch: [ 100/ 404] | Total Time: 1h 32m 49s
d_loss: 1.3867, g_loss: 46.5649, const_loss: 0.0022, l1_loss: 37.9605, fm_loss: 0.0107, perc_loss: 7.7054, edge: 0.1927
Epoch: [ 6], Batch: [ 200/ 404] | Total Time: 1h 36m 30s
d_loss: 1.3872, g_loss: 45.9773, const_loss: 0.0021, l1_loss: 37.6597, fm_loss: 0.0101, perc_loss: 7.4250, edge: 0.1870
Epoch: [ 6], Batch: [ 300/ 404] | Total Time: 1h 40m 11s
d_loss: 1.3867, g_loss: 45.2130, const_loss: 0.0021, l1_loss: 36.6534, fm_loss: 0.0099, perc_loss: 7.6725, edge: 0.1818
Epoch: [ 6], Batch: [ 400/ 404] | Total Time: 1h 43m 52s
d_loss: 1.3868, g_loss: 46.4315, const_loss: 0.0017, l1_loss: 37.2720, fm_loss: 0.0100, perc_loss: 8.2621, edge: 0.1923
--- End of Epoch 6 --- Time: 891.0s ---
LR Scheduler stepped. Current LR G: 0.000218, LR D: 0.000218
Epoch: [ 7], Batch: [ 0/ 404] | Total Time: 1h 44m 0s
d_loss: 1.3870, g_loss: 44.4468, const_loss: 0.0019, l1_loss: 36.0658, fm_loss: 0.0093, perc_loss: 7.4840, edge: 0.1929
Epoch: [ 7], Batch: [ 100/ 404] | Total Time: 1h 47m 41s
d_loss: 1.3868, g_loss: 45.9521, const_loss: 0.0019, l1_loss: 37.5713, fm_loss: 0.0103, perc_loss: 7.4863, edge: 0.1890
Epoch: [ 7], Batch: [ 200/ 404] | Total Time: 1h 51m 21s
d_loss: 1.3868, g_loss: 47.3385, const_loss: 0.0019, l1_loss: 38.9655, fm_loss: 0.0106, perc_loss: 7.4799, edge: 0.1873
Epoch: [ 7], Batch: [ 300/ 404] | Total Time: 1h 55m 2s
d_loss: 1.3868, g_loss: 45.0027, const_loss: 0.0018, l1_loss: 36.2265, fm_loss: 0.0092, perc_loss: 7.8861, edge: 0.1858
Epoch: [ 7], Batch: [ 400/ 404] | Total Time: 1h 58m 43s
d_loss: 1.3877, g_loss: 45.2559, const_loss: 0.0015, l1_loss: 36.9653, fm_loss: 0.0089, perc_loss: 7.4099, edge: 0.1770
--- End of Epoch 7 --- Time: 891.8s ---
LR Scheduler stepped. Current LR G: 0.000213, LR D: 0.000213
Epoch: [ 8], Batch: [ 0/ 404] | Total Time: 1h 58m 51s
d_loss: 1.3868, g_loss: 45.8112, const_loss: 0.0015, l1_loss: 37.4060, fm_loss: 0.0093, perc_loss: 7.5193, edge: 0.1818
Epoch: [ 8], Batch: [ 100/ 404] | Total Time: 2h 2m 32s
d_loss: 1.3871, g_loss: 45.9174, const_loss: 0.0018, l1_loss: 37.7680, fm_loss: 0.0096, perc_loss: 7.2640, edge: 0.1806
Epoch: [ 8], Batch: [ 200/ 404] | Total Time: 2h 6m 13s
d_loss: 1.3867, g_loss: 44.7717, const_loss: 0.0017, l1_loss: 36.3792, fm_loss: 0.0090, perc_loss: 7.5183, edge: 0.1702
Epoch: [ 8], Batch: [ 300/ 404] | Total Time: 2h 9m 54s
d_loss: 1.3870, g_loss: 47.4228, const_loss: 0.0017, l1_loss: 39.6168, fm_loss: 0.0117, perc_loss: 6.9352, edge: 0.1640
Epoch: [ 8], Batch: [ 400/ 404] | Total Time: 2h 13m 34s
d_loss: 1.3868, g_loss: 46.0952, const_loss: 0.0016, l1_loss: 37.2750, fm_loss: 0.0105, perc_loss: 7.9197, edge: 0.1950
--- End of Epoch 8 --- Time: 890.6s ---
LR Scheduler stepped. Current LR G: 0.000207, LR D: 0.000207
Epoch: [ 9], Batch: [ 0/ 404] | Total Time: 2h 13m 42s
d_loss: 1.3869, g_loss: 45.4242, const_loss: 0.0016, l1_loss: 37.3760, fm_loss: 0.0094, perc_loss: 7.1765, edge: 0.1672
Epoch: [ 9], Batch: [ 100/ 404] | Total Time: 2h 17m 22s
d_loss: 1.3872, g_loss: 44.0725, const_loss: 0.0019, l1_loss: 35.5708, fm_loss: 0.0096, perc_loss: 7.6091, edge: 0.1877
Epoch: [ 9], Batch: [ 200/ 404] | Total Time: 2h 21m 3s
d_loss: 1.3868, g_loss: 45.5713, const_loss: 0.0015, l1_loss: 37.0596, fm_loss: 0.0105, perc_loss: 7.6200, edge: 0.1863
Epoch: [ 9], Batch: [ 300/ 404] | Total Time: 2h 24m 43s
d_loss: 1.3870, g_loss: 46.0174, const_loss: 0.0016, l1_loss: 37.8080, fm_loss: 0.0098, perc_loss: 7.3169, edge: 0.1877
Epoch: [ 9], Batch: [ 400/ 404] | Total Time: 2h 28m 30s
d_loss: 1.3868, g_loss: 45.2284, const_loss: 0.0018, l1_loss: 36.9368, fm_loss: 0.0091, perc_loss: 7.4129, edge: 0.1744
--- End of Epoch 9 --- Time: 895.8s ---
LR Scheduler stepped. Current LR G: 0.000201, LR D: 0.000201
Epoch: [ 10], Batch: [ 0/ 404] | Total Time: 2h 28m 38s
d_loss: 1.3867, g_loss: 45.4000, const_loss: 0.0018, l1_loss: 36.7599, fm_loss: 0.0098, perc_loss: 7.7502, edge: 0.1850
Epoch: [ 10], Batch: [ 100/ 404] | Total Time: 2h 32m 18s
d_loss: 1.3868, g_loss: 45.8967, const_loss: 0.0016, l1_loss: 37.9645, fm_loss: 0.0096, perc_loss: 7.0503, edge: 0.1773
Epoch: [ 10], Batch: [ 200/ 404] | Total Time: 2h 36m 7s
d_loss: 1.3867, g_loss: 45.0293, const_loss: 0.0013, l1_loss: 37.1594, fm_loss: 0.0086, perc_loss: 7.0016, edge: 0.1650
Epoch: [ 10], Batch: [ 300/ 404] | Total Time: 2h 39m 48s
d_loss: 1.3869, g_loss: 44.8804, const_loss: 0.0013, l1_loss: 37.0668, fm_loss: 0.0097, perc_loss: 6.9420, edge: 0.1672
Epoch: [ 10], Batch: [ 400/ 404] | Total Time: 2h 43m 37s
d_loss: 1.3869, g_loss: 45.0564, const_loss: 0.0010, l1_loss: 37.3850, fm_loss: 0.0088, perc_loss: 6.8054, edge: 0.1628
--- End of Epoch 10 --- Time: 907.2s ---
LR Scheduler stepped. Current LR G: 0.000194, LR D: 0.000194
Epoch: [ 11], Batch: [ 0/ 404] | Total Time: 2h 43m 45s
d_loss: 1.3868, g_loss: 45.2792, const_loss: 0.0017, l1_loss: 36.9693, fm_loss: 0.0107, perc_loss: 7.4252, edge: 0.1790
Epoch: [ 11], Batch: [ 100/ 404] | Total Time: 2h 47m 26s
d_loss: 1.3868, g_loss: 44.8783, const_loss: 0.0012, l1_loss: 36.3218, fm_loss: 0.0092, perc_loss: 7.6606, edge: 0.1921
Epoch: [ 11], Batch: [ 200/ 404] | Total Time: 2h 51m 12s
d_loss: 1.3868, g_loss: 45.8806, const_loss: 0.0016, l1_loss: 38.4623, fm_loss: 0.0093, perc_loss: 6.5438, edge: 0.1703
from 4366 to 4368
Model 4366 loaded successfully
unpickled total 8072 examples
Starting training from epoch 0/39...
Epoch: [ 0], Batch: [ 0/ 404] | Total Time: 4s
d_loss: 1.3868, g_loss: 45.6566, const_loss: 0.0017, l1_loss: 37.1829, fm_loss: 0.0097, perc_loss: 7.5835, edge: 0.1853
Checkpoint step 100 reached, but saving starts after step 200.
Epoch: [ 0], Batch: [ 100/ 404] | Total Time: 3m 38s
d_loss: 1.3867, g_loss: 46.6532, const_loss: 0.0014, l1_loss: 38.2248, fm_loss: 0.0094, perc_loss: 7.5405, edge: 0.1838
Epoch: [ 0], Batch: [ 200/ 404] | Total Time: 7m 22s
d_loss: 1.3867, g_loss: 44.3632, const_loss: 0.0012, l1_loss: 35.9878, fm_loss: 0.0090, perc_loss: 7.4857, edge: 0.1862
Epoch: [ 0], Batch: [ 300/ 404] | Total Time: 11m 1s
d_loss: 1.3869, g_loss: 45.1879, const_loss: 0.0012, l1_loss: 37.0330, fm_loss: 0.0094, perc_loss: 7.2865, edge: 0.1645
Epoch: [ 0], Batch: [ 400/ 404] | Total Time: 14m 40s
d_loss: 1.3868, g_loss: 45.2913, const_loss: 0.0014, l1_loss: 37.0489, fm_loss: 0.0088, perc_loss: 7.3548, edge: 0.1842
--- End of Epoch 0 --- Time: 886.0s ---
LR Scheduler stepped. Current LR G: 0.000225, LR D: 0.000225
Epoch: [ 1], Batch: [ 0/ 404] | Total Time: 14m 48s
d_loss: 1.3867, g_loss: 45.8028, const_loss: 0.0015, l1_loss: 37.4104, fm_loss: 0.0098, perc_loss: 7.4965, edge: 0.1911
Epoch: [ 1], Batch: [ 100/ 404] | Total Time: 18m 26s
d_loss: 1.3868, g_loss: 44.8919, const_loss: 0.0017, l1_loss: 36.5353, fm_loss: 0.0099, perc_loss: 7.4715, edge: 0.1801
Epoch: [ 1], Batch: [ 200/ 404] | Total Time: 22m 5s
d_loss: 1.3877, g_loss: 46.7395, const_loss: 0.0010, l1_loss: 39.2617, fm_loss: 0.0117, perc_loss: 6.6083, edge: 0.1633
Epoch: [ 1], Batch: [ 300/ 404] | Total Time: 25m 44s
d_loss: 1.3867, g_loss: 45.7676, const_loss: 0.0017, l1_loss: 36.7109, fm_loss: 0.0093, perc_loss: 8.1624, edge: 0.1899
Epoch: [ 1], Batch: [ 400/ 404] | Total Time: 29m 22s
d_loss: 1.3868, g_loss: 45.3696, const_loss: 0.0013, l1_loss: 37.3220, fm_loss: 0.0089, perc_loss: 7.1633, edge: 0.1806
--- End of Epoch 1 --- Time: 882.3s ---
LR Scheduler stepped. Current LR G: 0.000224, LR D: 0.000224
Epoch: [ 2], Batch: [ 0/ 404] | Total Time: 29m 30s
d_loss: 1.3868, g_loss: 45.7790, const_loss: 0.0012, l1_loss: 38.3115, fm_loss: 0.0093, perc_loss: 6.5994, edge: 0.1642
Epoch: [ 2], Batch: [ 100/ 404] | Total Time: 33m 8s
d_loss: 1.3870, g_loss: 45.2587, const_loss: 0.0016, l1_loss: 36.6845, fm_loss: 0.0091, perc_loss: 7.6716, edge: 0.1986
Epoch: [ 2], Batch: [ 200/ 404] | Total Time: 36m 46s
d_loss: 1.3874, g_loss: 45.5661, const_loss: 0.0013, l1_loss: 38.0980, fm_loss: 0.0082, perc_loss: 6.6112, edge: 0.1540
Epoch: [ 2], Batch: [ 300/ 404] | Total Time: 40m 25s
d_loss: 1.3873, g_loss: 44.1656, const_loss: 0.0011, l1_loss: 36.0661, fm_loss: 0.0081, perc_loss: 7.2178, edge: 0.1791
Epoch: [ 2], Batch: [ 400/ 404] | Total Time: 44m 6s
d_loss: 1.3876, g_loss: 44.6390, const_loss: 0.0009, l1_loss: 36.0707, fm_loss: 0.0082, perc_loss: 7.6705, edge: 0.1953
--- End of Epoch 2 --- Time: 883.3s ---
LR Scheduler stepped. Current LR G: 0.000222, LR D: 0.000222
Epoch: [ 3], Batch: [ 0/ 404] | Total Time: 44m 13s
d_loss: 1.3881, g_loss: 45.7792, const_loss: 0.0016, l1_loss: 38.5020, fm_loss: 0.0082, perc_loss: 6.4123, edge: 0.1616
Epoch: [ 3], Batch: [ 100/ 404] | Total Time: 47m 53s
d_loss: 1.3869, g_loss: 45.8382, const_loss: 0.0016, l1_loss: 38.3416, fm_loss: 0.0089, perc_loss: 6.6165, edge: 0.1762
Epoch: [ 3], Batch: [ 200/ 404] | Total Time: 51m 31s
d_loss: 1.3869, g_loss: 44.1660, const_loss: 0.0013, l1_loss: 36.2489, fm_loss: 0.0080, perc_loss: 7.0355, edge: 0.1790
Epoch: [ 3], Batch: [ 300/ 404] | Total Time: 55m 11s
d_loss: 1.3867, g_loss: 45.0707, const_loss: 0.0013, l1_loss: 37.0570, fm_loss: 0.0090, perc_loss: 7.1443, edge: 0.1658
Epoch: [ 3], Batch: [ 400/ 404] | Total Time: 58m 50s
d_loss: 1.3868, g_loss: 44.6572, const_loss: 0.0014, l1_loss: 36.7198, fm_loss: 0.0091, perc_loss: 7.0711, edge: 0.1625
--- End of Epoch 3 --- Time: 884.5s ---
LR Scheduler stepped. Current LR G: 0.000220, LR D: 0.000220
Epoch: [ 4], Batch: [ 0/ 404] | Total Time: 58m 58s
d_loss: 1.3868, g_loss: 44.5943, const_loss: 0.0015, l1_loss: 36.5974, fm_loss: 0.0085, perc_loss: 7.1371, edge: 0.1564
Epoch: [ 4], Batch: [ 100/ 404] | Total Time: 1h 2m 39s
d_loss: 1.3869, g_loss: 45.3000, const_loss: 0.0011, l1_loss: 37.4935, fm_loss: 0.0090, perc_loss: 6.9342, edge: 0.1688
Epoch: [ 4], Batch: [ 200/ 404] | Total Time: 1h 6m 17s
d_loss: 1.3868, g_loss: 44.7900, const_loss: 0.0013, l1_loss: 36.8500, fm_loss: 0.0082, perc_loss: 7.0666, edge: 0.1706
Epoch: [ 4], Batch: [ 300/ 404] | Total Time: 1h 9m 55s
d_loss: 1.3870, g_loss: 45.7650, const_loss: 0.0013, l1_loss: 37.8393, fm_loss: 0.0084, perc_loss: 7.0481, edge: 0.1746
Epoch: [ 4], Batch: [ 400/ 404] | Total Time: 1h 13m 36s
d_loss: 1.3868, g_loss: 44.3453, const_loss: 0.0013, l1_loss: 36.7337, fm_loss: 0.0095, perc_loss: 6.7323, edge: 0.1751
--- End of Epoch 4 --- Time: 885.7s ---
LR Scheduler stepped. Current LR G: 0.000216, LR D: 0.000216
Epoch: [ 5], Batch: [ 0/ 404] | Total Time: 1h 13m 43s
d_loss: 1.3873, g_loss: 45.3333, const_loss: 0.0013, l1_loss: 36.8390, fm_loss: 0.0091, perc_loss: 7.6264, edge: 0.1642
Epoch: [ 5], Batch: [ 100/ 404] | Total Time: 1h 17m 22s
d_loss: 1.3869, g_loss: 46.6879, const_loss: 0.0012, l1_loss: 39.4324, fm_loss: 0.0097, perc_loss: 6.3973, edge: 0.1538
Epoch: [ 5], Batch: [ 200/ 404] | Total Time: 1h 21m 2s
d_loss: 1.3868, g_loss: 45.1666, const_loss: 0.0011, l1_loss: 37.8331, fm_loss: 0.0084, perc_loss: 6.4850, edge: 0.1455
Epoch: [ 5], Batch: [ 300/ 404] | Total Time: 1h 24m 40s
d_loss: 1.3868, g_loss: 45.9261, const_loss: 0.0012, l1_loss: 38.3948, fm_loss: 0.0089, perc_loss: 6.6627, edge: 0.1652
Epoch: [ 5], Batch: [ 400/ 404] | Total Time: 1h 28m 17s
d_loss: 1.3868, g_loss: 44.3240, const_loss: 0.0011, l1_loss: 36.3100, fm_loss: 0.0081, perc_loss: 7.1292, edge: 0.1824
--- End of Epoch 5 --- Time: 881.8s ---
LR Scheduler stepped. Current LR G: 0.000213, LR D: 0.000213
Epoch: [ 6], Batch: [ 0/ 404] | Total Time: 1h 28m 25s
d_loss: 1.3868, g_loss: 45.9917, const_loss: 0.0013, l1_loss: 38.4433, fm_loss: 0.0093, perc_loss: 6.6733, edge: 0.1711
Epoch: [ 6], Batch: [ 100/ 404] | Total Time: 1h 32m 4s
d_loss: 1.3868, g_loss: 45.2222, const_loss: 0.0012, l1_loss: 37.4184, fm_loss: 0.0087, perc_loss: 6.9394, edge: 0.1611
Epoch: [ 6], Batch: [ 200/ 404] | Total Time: 1h 35m 42s
d_loss: 1.3869, g_loss: 45.1402, const_loss: 0.0014, l1_loss: 37.4718, fm_loss: 0.0084, perc_loss: 6.8178, edge: 0.1474
Epoch: [ 6], Batch: [ 300/ 404] | Total Time: 1h 39m 21s
d_loss: 1.3868, g_loss: 43.9844, const_loss: 0.0013, l1_loss: 36.1405, fm_loss: 0.0081, perc_loss: 6.9695, edge: 0.1717
Epoch: [ 6], Batch: [ 400/ 404] | Total Time: 1h 42m 59s
d_loss: 1.3868, g_loss: 44.5656, const_loss: 0.0012, l1_loss: 36.4330, fm_loss: 0.0082, perc_loss: 7.2638, edge: 0.1659
--- End of Epoch 6 --- Time: 881.2s ---
LR Scheduler stepped. Current LR G: 0.000208, LR D: 0.000208
Epoch: [ 7], Batch: [ 0/ 404] | Total Time: 1h 43m 6s
d_loss: 1.3869, g_loss: 43.2027, const_loss: 0.0015, l1_loss: 35.6316, fm_loss: 0.0078, perc_loss: 6.7091, edge: 0.1592
Epoch: [ 7], Batch: [ 100/ 404] | Total Time: 1h 46m 45s
d_loss: 1.3867, g_loss: 44.3842, const_loss: 0.0011, l1_loss: 36.9526, fm_loss: 0.0083, perc_loss: 6.5746, edge: 0.1543
Epoch: [ 7], Batch: [ 200/ 404] | Total Time: 1h 50m 24s
d_loss: 1.3868, g_loss: 45.6226, const_loss: 0.0014, l1_loss: 38.1496, fm_loss: 0.0081, perc_loss: 6.6183, edge: 0.1519
Epoch: [ 7], Batch: [ 300/ 404] | Total Time: 1h 54m 4s
d_loss: 1.3868, g_loss: 44.2472, const_loss: 0.0013, l1_loss: 36.2361, fm_loss: 0.0080, perc_loss: 7.1319, edge: 0.1766
Epoch: [ 7], Batch: [ 400/ 404] | Total Time: 1h 57m 43s
d_loss: 1.4033, g_loss: 44.2049, const_loss: 0.0009, l1_loss: 36.5083, fm_loss: 0.0096, perc_loss: 6.8093, edge: 0.1833
--- End of Epoch 7 --- Time: 884.1s ---
LR Scheduler stepped. Current LR G: 0.000204, LR D: 0.000204
Epoch: [ 8], Batch: [ 0/ 404] | Total Time: 1h 57m 50s
d_loss: 1.3870, g_loss: 45.2161, const_loss: 0.0010, l1_loss: 37.3458, fm_loss: 0.0094, perc_loss: 6.9994, edge: 0.1672
Epoch: [ 8], Batch: [ 100/ 404] | Total Time: 2h 1m 29s
d_loss: 1.3869, g_loss: 44.5756, const_loss: 0.0014, l1_loss: 37.2014, fm_loss: 0.0085, perc_loss: 6.5188, edge: 0.1521
Epoch: [ 8], Batch: [ 200/ 404] | Total Time: 2h 5m 7s
d_loss: 1.3868, g_loss: 44.1188, const_loss: 0.0012, l1_loss: 36.2001, fm_loss: 0.0087, perc_loss: 7.0404, edge: 0.1750
Epoch: [ 8], Batch: [ 300/ 404] | Total Time: 2h 8m 45s
d_loss: 1.3868, g_loss: 46.3739, const_loss: 0.0011, l1_loss: 39.2476, fm_loss: 0.0100, perc_loss: 6.2589, edge: 0.1630
Epoch: [ 8], Batch: [ 400/ 404] | Total Time: 2h 12m 23s
d_loss: 1.3868, g_loss: 44.1225, const_loss: 0.0010, l1_loss: 36.3447, fm_loss: 0.0083, perc_loss: 6.9080, edge: 0.1671
--- End of Epoch 8 --- Time: 880.8s ---
LR Scheduler stepped. Current LR G: 0.000198, LR D: 0.000198
Epoch: [ 9], Batch: [ 0/ 404] | Total Time: 2h 12m 31s
d_loss: 1.3870, g_loss: 43.9846, const_loss: 0.0010, l1_loss: 36.7297, fm_loss: 0.0079, perc_loss: 6.3852, edge: 0.1675
Epoch: [ 9], Batch: [ 100/ 404] | Total Time: 2h 16m 9s
d_loss: 1.3869, g_loss: 42.8964, const_loss: 0.0012, l1_loss: 35.1387, fm_loss: 0.0082, perc_loss: 6.8836, edge: 0.1712
Epoch: [ 9], Batch: [ 200/ 404] | Total Time: 2h 19m 47s
d_loss: 1.3868, g_loss: 43.9584, const_loss: 0.0008, l1_loss: 36.3238, fm_loss: 0.0086, perc_loss: 6.7597, edge: 0.1720
Epoch: [ 9], Batch: [ 300/ 404] | Total Time: 2h 23m 25s
d_loss: 1.3868, g_loss: 44.6966, const_loss: 0.0011, l1_loss: 37.3188, fm_loss: 0.0080, perc_loss: 6.5300, edge: 0.1453
Epoch: [ 9], Batch: [ 400/ 404] | Total Time: 2h 27m 3s
d_loss: 1.3867, g_loss: 43.9369, const_loss: 0.0013, l1_loss: 36.4409, fm_loss: 0.0077, perc_loss: 6.6245, edge: 0.1692
--- End of Epoch 9 --- Time: 879.7s ---
LR Scheduler stepped. Current LR G: 0.000192, LR D: 0.000192
Epoch: [ 10], Batch: [ 0/ 404] | Total Time: 2h 27m 11s
d_loss: 1.3867, g_loss: 44.1115, const_loss: 0.0011, l1_loss: 36.2772, fm_loss: 0.0081, perc_loss: 6.9703, edge: 0.1614
Epoch: [ 10], Batch: [ 100/ 404] | Total Time: 2h 30m 49s
d_loss: 1.3868, g_loss: 44.6209, const_loss: 0.0012, l1_loss: 37.4423, fm_loss: 0.0076, perc_loss: 6.3202, edge: 0.1563
Epoch: [ 10], Batch: [ 200/ 404] | Total Time: 2h 34m 27s
d_loss: 1.3867, g_loss: 43.9126, const_loss: 0.0009, l1_loss: 36.7213, fm_loss: 0.0073, perc_loss: 6.3281, edge: 0.1617
Epoch: [ 10], Batch: [ 300/ 404] | Total Time: 2h 38m 8s
d_loss: 1.3868, g_loss: 44.0946, const_loss: 0.0009, l1_loss: 36.8236, fm_loss: 0.0081, perc_loss: 6.4149, edge: 0.1538
Epoch: [ 10], Batch: [ 400/ 404] | Total Time: 2h 41m 46s
d_loss: 1.3869, g_loss: 44.2050, const_loss: 0.0007, l1_loss: 37.0711, fm_loss: 0.0067, perc_loss: 6.2774, edge: 0.1557
--- End of Epoch 10 --- Time: 882.9s ---
LR Scheduler stepped. Current LR G: 0.000186, LR D: 0.000186
Epoch: [ 11], Batch: [ 0/ 404] | Total Time: 2h 41m 54s
d_loss: 1.3868, g_loss: 44.3260, const_loss: 0.0011, l1_loss: 36.5962, fm_loss: 0.0082, perc_loss: 6.8694, edge: 0.1578
Epoch: [ 11], Batch: [ 100/ 404] | Total Time: 2h 45m 32s
d_loss: 1.3868, g_loss: 43.3058, const_loss: 0.0010, l1_loss: 35.6602, fm_loss: 0.0073, perc_loss: 6.7766, edge: 0.1674
Epoch: [ 11], Batch: [ 200/ 404] | Total Time: 2h 49m 10s
d_loss: 1.3867, g_loss: 45.0930, const_loss: 0.0011, l1_loss: 38.2575, fm_loss: 0.0073, perc_loss: 5.9934, edge: 0.1404
Epoch: [ 11], Batch: [ 300/ 404] | Total Time: 2h 52m 50s
d_loss: 1.3868, g_loss: 45.1709, const_loss: 0.0012, l1_loss: 37.8256, fm_loss: 0.0084, perc_loss: 6.4928, edge: 0.1496
Epoch: [ 11], Batch: [ 400/ 404] | Total Time: 2h 56m 28s
d_loss: 1.3870, g_loss: 45.4194, const_loss: 0.0009, l1_loss: 37.6699, fm_loss: 0.0083, perc_loss: 6.8768, edge: 0.1702
--- End of Epoch 11 --- Time: 881.7s ---
LR Scheduler stepped. Current LR G: 0.000179, LR D: 0.000179
Epoch: [ 12], Batch: [ 0/ 404] | Total Time: 2h 56m 36s
d_loss: 1.3868, g_loss: 43.4101, const_loss: 0.0008, l1_loss: 36.0527, fm_loss: 0.0072, perc_loss: 6.4910, edge: 0.1650
Epoch: [ 12], Batch: [ 100/ 404] | Total Time: 3h 15s
d_loss: 1.3868, g_loss: 42.9837, const_loss: 0.0008, l1_loss: 35.6439, fm_loss: 0.0075, perc_loss: 6.4858, edge: 0.1523
Epoch: [ 12], Batch: [ 200/ 404] | Total Time: 3h 3m 53s
d_loss: 1.3868, g_loss: 43.1683, const_loss: 0.0009, l1_loss: 35.7422, fm_loss: 0.0077, perc_loss: 6.5809, edge: 0.1432
Epoch: [ 12], Batch: [ 300/ 404] | Total Time: 3h 7m 32s
d_loss: 1.3870, g_loss: 43.6023, const_loss: 0.0010, l1_loss: 36.1851, fm_loss: 0.0081, perc_loss: 6.5594, edge: 0.1555
Epoch: [ 12], Batch: [ 400/ 404] | Total Time: 3h 11m 10s
d_loss: 1.3868, g_loss: 45.4847, const_loss: 0.0010, l1_loss: 38.9365, fm_loss: 0.0082, perc_loss: 5.6975, edge: 0.1482
--- End of Epoch 12 --- Time: 882.2s ---
LR Scheduler stepped. Current LR G: 0.000172, LR D: 0.000172
Epoch: [ 13], Batch: [ 0/ 404] | Total Time: 3h 11m 18s
d_loss: 1.3867, g_loss: 44.0985, const_loss: 0.0009, l1_loss: 36.8137, fm_loss: 0.0072, perc_loss: 6.4367, edge: 0.1466
Epoch: [ 13], Batch: [ 100/ 404] | Total Time: 3h 14m 56s
d_loss: 1.3867, g_loss: 44.4137, const_loss: 0.0009, l1_loss: 37.0712, fm_loss: 0.0081, perc_loss: 6.4923, edge: 0.1479
Epoch: [ 13], Batch: [ 200/ 404] | Total Time: 3h 18m 34s
d_loss: 1.3868, g_loss: 43.5688, const_loss: 0.0008, l1_loss: 36.5640, fm_loss: 0.0070, perc_loss: 6.1646, edge: 0.1389
Epoch: [ 13], Batch: [ 300/ 404] | Total Time: 3h 22m 12s
d_loss: 1.3868, g_loss: 43.0508, const_loss: 0.0007, l1_loss: 35.7255, fm_loss: 0.0069, perc_loss: 6.4790, edge: 0.1454
Epoch: [ 13], Batch: [ 400/ 404] | Total Time: 3h 25m 52s
d_loss: 1.3868, g_loss: 43.4664, const_loss: 0.0008, l1_loss: 36.0616, fm_loss: 0.0071, perc_loss: 6.5600, edge: 0.1436
--- End of Epoch 13 --- Time: 881.6s ---
LR Scheduler stepped. Current LR G: 0.000164, LR D: 0.000164
Epoch: [ 14], Batch: [ 0/ 404] | Total Time: 3h 25m 59s
d_loss: 1.3872, g_loss: 42.2635, const_loss: 0.0008, l1_loss: 34.3467, fm_loss: 0.0073, perc_loss: 7.0496, edge: 0.1658
Epoch: [ 14], Batch: [ 100/ 404] | Total Time: 3h 29m 37s
d_loss: 1.3868, g_loss: 43.9078, const_loss: 0.0007, l1_loss: 36.9432, fm_loss: 0.0069, perc_loss: 6.1027, edge: 0.1610
from 4368 to 4370
Model 4368 loaded successfully
unpickled total 8072 examples
Starting training from epoch 0/39...
Epoch: [ 0], Batch: [ 0/ 404] | Total Time: 4s
d_loss: 1.3868, g_loss: 43.3545, const_loss: 0.0009, l1_loss: 36.1095, fm_loss: 0.0071, perc_loss: 6.3958, edge: 0.1478
Checkpoint step 100 reached, but saving starts after step 200.
Epoch: [ 0], Batch: [ 100/ 404] | Total Time: 3m 43s
d_loss: 1.3871, g_loss: 44.5041, const_loss: 0.0009, l1_loss: 37.2622, fm_loss: 0.0066, perc_loss: 6.3748, edge: 0.1663
Epoch: [ 0], Batch: [ 200/ 404] | Total Time: 7m 32s
d_loss: 1.3868, g_loss: 42.7257, const_loss: 0.0008, l1_loss: 35.3905, fm_loss: 0.0068, perc_loss: 6.4828, edge: 0.1515
Epoch: [ 0], Batch: [ 300/ 404] | Total Time: 11m 15s
d_loss: 1.3868, g_loss: 43.9863, const_loss: 0.0007, l1_loss: 36.5599, fm_loss: 0.0073, perc_loss: 6.5629, edge: 0.1621
Epoch: [ 0], Batch: [ 400/ 404] | Total Time: 14m 57s
d_loss: 1.3867, g_loss: 43.6766, const_loss: 0.0008, l1_loss: 36.4424, fm_loss: 0.0068, perc_loss: 6.3788, edge: 0.1544
--- End of Epoch 0 --- Time: 903.7s ---
LR Scheduler stepped. Current LR G: 0.000210, LR D: 0.000210
Epoch: [ 1], Batch: [ 0/ 404] | Total Time: 15m 5s
d_loss: 1.3867, g_loss: 43.9514, const_loss: 0.0009, l1_loss: 36.5840, fm_loss: 0.0073, perc_loss: 6.4964, edge: 0.1695
Epoch: [ 1], Batch: [ 100/ 404] | Total Time: 18m 49s
d_loss: 1.3867, g_loss: 43.8949, const_loss: 0.0014, l1_loss: 36.1866, fm_loss: 0.0076, perc_loss: 6.8373, edge: 0.1687
Epoch: [ 1], Batch: [ 200/ 404] | Total Time: 22m 39s
d_loss: 1.3875, g_loss: 46.3380, const_loss: 0.0006, l1_loss: 39.2029, fm_loss: 0.0089, perc_loss: 6.2772, edge: 0.1549
Epoch: [ 1], Batch: [ 300/ 404] | Total Time: 26m 22s
d_loss: 1.3867, g_loss: 42.8528, const_loss: 0.0009, l1_loss: 35.2915, fm_loss: 0.0069, perc_loss: 6.6965, edge: 0.1637
Epoch: [ 1], Batch: [ 400/ 404] | Total Time: 30m 5s
d_loss: 1.3868, g_loss: 43.4925, const_loss: 0.0010, l1_loss: 36.5911, fm_loss: 0.0066, perc_loss: 6.0604, edge: 0.1401
--- End of Epoch 1 --- Time: 907.7s ---
LR Scheduler stepped. Current LR G: 0.000209, LR D: 0.000209
Epoch: [ 2], Batch: [ 0/ 404] | Total Time: 30m 13s
d_loss: 1.3869, g_loss: 44.1878, const_loss: 0.0009, l1_loss: 37.6481, fm_loss: 0.0069, perc_loss: 5.6935, edge: 0.1450
Epoch: [ 2], Batch: [ 100/ 404] | Total Time: 33m 56s
d_loss: 1.3868, g_loss: 43.1566, const_loss: 0.0010, l1_loss: 35.7248, fm_loss: 0.0068, perc_loss: 6.5753, edge: 0.1553
Epoch: [ 2], Batch: [ 200/ 404] | Total Time: 37m 40s
d_loss: 1.3871, g_loss: 44.4738, const_loss: 0.0009, l1_loss: 37.7087, fm_loss: 0.0065, perc_loss: 5.9203, edge: 0.1441
Epoch: [ 2], Batch: [ 300/ 404] | Total Time: 41m 23s
d_loss: 1.3871, g_loss: 44.4883, const_loss: 0.0008, l1_loss: 36.4237, fm_loss: 0.0069, perc_loss: 7.1601, edge: 0.2035
Epoch: [ 2], Batch: [ 400/ 404] | Total Time: 45m 12s
d_loss: 1.3868, g_loss: 42.8969, const_loss: 0.0008, l1_loss: 35.3759, fm_loss: 0.0063, perc_loss: 6.6735, edge: 0.1472
--- End of Epoch 2 --- Time: 906.5s ---
LR Scheduler stepped. Current LR G: 0.000207, LR D: 0.000207
Epoch: [ 3], Batch: [ 0/ 404] | Total Time: 45m 20s
d_loss: 1.3872, g_loss: 44.5730, const_loss: 0.0011, l1_loss: 38.0540, fm_loss: 0.0066, perc_loss: 5.6831, edge: 0.1347
Epoch: [ 3], Batch: [ 100/ 404] | Total Time: 49m 3s
d_loss: 1.3867, g_loss: 44.9036, const_loss: 0.0011, l1_loss: 38.0266, fm_loss: 0.0071, perc_loss: 6.0423, edge: 0.1331
Epoch: [ 3], Batch: [ 200/ 404] | Total Time: 52m 46s
d_loss: 1.3868, g_loss: 43.7339, const_loss: 0.0009, l1_loss: 36.2928, fm_loss: 0.0066, perc_loss: 6.5796, edge: 0.1606
Epoch: [ 3], Batch: [ 300/ 404] | Total Time: 56m 35s
d_loss: 1.3867, g_loss: 44.1288, const_loss: 0.0009, l1_loss: 36.7218, fm_loss: 0.0069, perc_loss: 6.5440, edge: 0.1619
Epoch: [ 3], Batch: [ 400/ 404] | Total Time: 1h 18s
d_loss: 1.3867, g_loss: 43.1910, const_loss: 0.0009, l1_loss: 36.0512, fm_loss: 0.0068, perc_loss: 6.2774, edge: 0.1614
--- End of Epoch 3 --- Time: 906.1s ---
LR Scheduler stepped. Current LR G: 0.000205, LR D: 0.000205
Epoch: [ 4], Batch: [ 0/ 404] | Total Time: 1h 26s
d_loss: 1.3867, g_loss: 43.5815, const_loss: 0.0009, l1_loss: 36.0451, fm_loss: 0.0065, perc_loss: 6.6582, edge: 0.1774
Epoch: [ 4], Batch: [ 100/ 404] | Total Time: 1h 4m 9s
d_loss: 1.3868, g_loss: 44.2843, const_loss: 0.0008, l1_loss: 37.1669, fm_loss: 0.0070, perc_loss: 6.2601, edge: 0.1562
Epoch: [ 4], Batch: [ 200/ 404] | Total Time: 1h 7m 52s
d_loss: 1.3867, g_loss: 43.2957, const_loss: 0.0008, l1_loss: 36.2078, fm_loss: 0.0065, perc_loss: 6.2368, edge: 0.1504
Epoch: [ 4], Batch: [ 300/ 404] | Total Time: 1h 11m 35s
d_loss: 1.3868, g_loss: 44.9040, const_loss: 0.0009, l1_loss: 37.6144, fm_loss: 0.0064, perc_loss: 6.4358, edge: 0.1532
Epoch: [ 4], Batch: [ 400/ 404] | Total Time: 1h 15m 17s
d_loss: 1.3869, g_loss: 43.7489, const_loss: 0.0008, l1_loss: 36.5023, fm_loss: 0.0069, perc_loss: 6.3956, edge: 0.1499
--- End of Epoch 4 --- Time: 899.9s ---
LR Scheduler stepped. Current LR G: 0.000202, LR D: 0.000202
Epoch: [ 5], Batch: [ 0/ 404] | Total Time: 1h 15m 26s
d_loss: 1.3872, g_loss: 43.6551, const_loss: 0.0008, l1_loss: 35.9866, fm_loss: 0.0066, perc_loss: 6.8158, edge: 0.1519
Epoch: [ 5], Batch: [ 100/ 404] | Total Time: 1h 19m 9s
d_loss: 1.3868, g_loss: 46.0896, const_loss: 0.0010, l1_loss: 39.2585, fm_loss: 0.0066, perc_loss: 6.0017, edge: 0.1285
Epoch: [ 5], Batch: [ 200/ 404] | Total Time: 1h 22m 57s
d_loss: 1.3868, g_loss: 44.9461, const_loss: 0.0008, l1_loss: 37.8096, fm_loss: 0.0064, perc_loss: 6.2819, edge: 0.1541
Epoch: [ 5], Batch: [ 300/ 404] | Total Time: 1h 26m 40s
d_loss: 1.3868, g_loss: 45.2992, const_loss: 0.0008, l1_loss: 38.2197, fm_loss: 0.0065, perc_loss: 6.2242, edge: 0.1547
Epoch: [ 5], Batch: [ 400/ 404] | Total Time: 1h 30m 28s
d_loss: 1.3867, g_loss: 43.0011, const_loss: 0.0007, l1_loss: 35.7632, fm_loss: 0.0063, perc_loss: 6.3819, edge: 0.1557
--- End of Epoch 5 --- Time: 910.6s ---
LR Scheduler stepped. Current LR G: 0.000199, LR D: 0.000199
Epoch: [ 6], Batch: [ 0/ 404] | Total Time: 1h 30m 36s
d_loss: 1.3868, g_loss: 44.8044, const_loss: 0.0009, l1_loss: 37.9478, fm_loss: 0.0068, perc_loss: 6.0264, edge: 0.1291
Epoch: [ 6], Batch: [ 100/ 404] | Total Time: 1h 34m 19s
d_loss: 1.3867, g_loss: 43.7896, const_loss: 0.0008, l1_loss: 36.7586, fm_loss: 0.0063, perc_loss: 6.1863, edge: 0.1443
Epoch: [ 6], Batch: [ 200/ 404] | Total Time: 1h 38m 2s
d_loss: 1.3870, g_loss: 43.4614, const_loss: 0.0010, l1_loss: 36.6630, fm_loss: 0.0061, perc_loss: 5.9521, edge: 0.1459
Epoch: [ 6], Batch: [ 300/ 404] | Total Time: 1h 41m 45s
d_loss: 1.3867, g_loss: 43.3659, const_loss: 0.0009, l1_loss: 35.9874, fm_loss: 0.0062, perc_loss: 6.5109, edge: 0.1671
Epoch: [ 6], Batch: [ 400/ 404] | Total Time: 1h 45m 32s
d_loss: 1.3867, g_loss: 43.8966, const_loss: 0.0008, l1_loss: 36.2056, fm_loss: 0.0063, perc_loss: 6.8253, edge: 0.1653
--- End of Epoch 6 --- Time: 903.8s ---
LR Scheduler stepped. Current LR G: 0.000195, LR D: 0.000195
Epoch: [ 7], Batch: [ 0/ 404] | Total Time: 1h 45m 40s
d_loss: 1.3870, g_loss: 41.9727, const_loss: 0.0010, l1_loss: 35.0808, fm_loss: 0.0059, perc_loss: 6.0291, edge: 0.1626
Epoch: [ 7], Batch: [ 100/ 404] | Total Time: 1h 49m 24s
d_loss: 1.3867, g_loss: 43.5877, const_loss: 0.0007, l1_loss: 36.5291, fm_loss: 0.0058, perc_loss: 6.2130, edge: 0.1456
Epoch: [ 7], Batch: [ 200/ 404] | Total Time: 1h 53m 11s
d_loss: 1.3869, g_loss: 44.4010, const_loss: 0.0011, l1_loss: 37.6947, fm_loss: 0.0061, perc_loss: 5.8700, edge: 0.1358
Epoch: [ 7], Batch: [ 300/ 404] | Total Time: 1h 56m 57s
d_loss: 1.3867, g_loss: 43.1836, const_loss: 0.0008, l1_loss: 35.7233, fm_loss: 0.0061, perc_loss: 6.6050, edge: 0.1550
Epoch: [ 7], Batch: [ 400/ 404] | Total Time: 2h 40s
d_loss: 1.3890, g_loss: 42.7758, const_loss: 0.0007, l1_loss: 35.9940, fm_loss: 0.0082, perc_loss: 5.9508, edge: 0.1287
--- End of Epoch 7 --- Time: 908.2s ---
LR Scheduler stepped. Current LR G: 0.000190, LR D: 0.000190
Epoch: [ 8], Batch: [ 0/ 404] | Total Time: 2h 48s
d_loss: 1.3868, g_loss: 45.2584, const_loss: 0.0008, l1_loss: 37.5102, fm_loss: 0.0074, perc_loss: 6.8930, edge: 0.1536
Epoch: [ 8], Batch: [ 100/ 404] | Total Time: 2h 4m 36s
d_loss: 1.3868, g_loss: 44.2865, const_loss: 0.0011, l1_loss: 37.2218, fm_loss: 0.0066, perc_loss: 6.2310, edge: 0.1327
Epoch: [ 8], Batch: [ 200/ 404] | Total Time: 2h 8m 20s
d_loss: 1.3867, g_loss: 43.1613, const_loss: 0.0007, l1_loss: 35.7802, fm_loss: 0.0068, perc_loss: 6.5353, edge: 0.1449
Epoch: [ 8], Batch: [ 300/ 404] | Total Time: 2h 12m 3s
d_loss: 1.3867, g_loss: 45.5215, const_loss: 0.0008, l1_loss: 38.9142, fm_loss: 0.0075, perc_loss: 5.7677, edge: 0.1380
Epoch: [ 8], Batch: [ 400/ 404] | Total Time: 2h 15m 50s
d_loss: 1.3868, g_loss: 43.3217, const_loss: 0.0008, l1_loss: 36.0197, fm_loss: 0.0064, perc_loss: 6.4546, edge: 0.1469
--- End of Epoch 8 --- Time: 909.8s ---
LR Scheduler stepped. Current LR G: 0.000185, LR D: 0.000185
Epoch: [ 9], Batch: [ 0/ 404] | Total Time: 2h 15m 58s
d_loss: 1.3869, g_loss: 42.6311, const_loss: 0.0008, l1_loss: 36.1898, fm_loss: 0.0058, perc_loss: 5.5957, edge: 0.1456
Epoch: [ 9], Batch: [ 100/ 404] | Total Time: 2h 19m 41s
d_loss: 1.3869, g_loss: 42.8904, const_loss: 0.0009, l1_loss: 35.3010, fm_loss: 0.0067, perc_loss: 6.7162, edge: 0.1723
Epoch: [ 9], Batch: [ 200/ 404] | Total Time: 2h 23m 37s
d_loss: 1.3868, g_loss: 42.9760, const_loss: 0.0006, l1_loss: 35.9255, fm_loss: 0.0063, perc_loss: 6.1981, edge: 0.1520
Epoch: [ 9], Batch: [ 300/ 404] | Total Time: 2h 27m 20s
d_loss: 1.3869, g_loss: 43.7174, const_loss: 0.0008, l1_loss: 36.9058, fm_loss: 0.0061, perc_loss: 5.9669, edge: 0.1445
Epoch: [ 9], Batch: [ 400/ 404] | Total Time: 2h 31m 4s
d_loss: 1.3868, g_loss: 42.7818, const_loss: 0.0010, l1_loss: 35.9166, fm_loss: 0.0059, perc_loss: 6.0126, edge: 0.1523
--- End of Epoch 9 --- Time: 914.2s ---
LR Scheduler stepped. Current LR G: 0.000179, LR D: 0.000179
Epoch: [ 10], Batch: [ 0/ 404] | Total Time: 2h 31m 12s
d_loss: 1.3867, g_loss: 43.2039, const_loss: 0.0009, l1_loss: 35.8833, fm_loss: 0.0065, perc_loss: 6.4523, edge: 0.1675
Epoch: [ 10], Batch: [ 100/ 404] | Total Time: 2h 34m 55s
d_loss: 1.3867, g_loss: 44.2265, const_loss: 0.0009, l1_loss: 37.2903, fm_loss: 0.0058, perc_loss: 6.0958, edge: 0.1403
Epoch: [ 10], Batch: [ 200/ 404] | Total Time: 2h 38m 42s
d_loss: 1.3868, g_loss: 43.1544, const_loss: 0.0007, l1_loss: 36.4408, fm_loss: 0.0054, perc_loss: 5.8719, edge: 0.1424
Epoch: [ 10], Batch: [ 300/ 404] | Total Time: 2h 42m 26s
d_loss: 1.3868, g_loss: 43.2432, const_loss: 0.0007, l1_loss: 36.4693, fm_loss: 0.0062, perc_loss: 5.9124, edge: 0.1612
Epoch: [ 10], Batch: [ 400/ 404] | Total Time: 2h 46m 9s
d_loss: 1.3868, g_loss: 43.4288, const_loss: 0.0005, l1_loss: 36.7781, fm_loss: 0.0052, perc_loss: 5.8039, edge: 0.1477
--- End of Epoch 10 --- Time: 905.0s ---
LR Scheduler stepped. Current LR G: 0.000173, LR D: 0.000173
Epoch: [ 11], Batch: [ 0/ 404] | Total Time: 2h 46m 17s
d_loss: 1.3867, g_loss: 43.5446, const_loss: 0.0009, l1_loss: 36.3089, fm_loss: 0.0065, perc_loss: 6.3670, edge: 0.1680
Epoch: [ 11], Batch: [ 100/ 404] | Total Time: 2h 50m 0s
d_loss: 1.3867, g_loss: 42.8885, const_loss: 0.0007, l1_loss: 35.5936, fm_loss: 0.0058, perc_loss: 6.4636, edge: 0.1315
Epoch: [ 11], Batch: [ 200/ 404] | Total Time: 2h 53m 43s
d_loss: 1.3868, g_loss: 43.9044, const_loss: 0.0009, l1_loss: 37.7052, fm_loss: 0.0057, perc_loss: 5.3574, edge: 0.1419
Epoch: [ 11], Batch: [ 300/ 404] | Total Time: 2h 57m 28s
d_loss: 1.3867, g_loss: 44.7979, const_loss: 0.0010, l1_loss: 37.7395, fm_loss: 0.0068, perc_loss: 6.2016, edge: 0.1557
Epoch: [ 11], Batch: [ 400/ 404] | Total Time: 3h 1m 11s
d_loss: 1.3869, g_loss: 44.1776, const_loss: 0.0008, l1_loss: 37.1906, fm_loss: 0.0063, perc_loss: 6.1366, edge: 0.1499
--- End of Epoch 11 --- Time: 901.7s ---
LR Scheduler stepped. Current LR G: 0.000167, LR D: 0.000167
Epoch: [ 12], Batch: [ 0/ 404] | Total Time: 3h 1m 19s
d_loss: 1.3868, g_loss: 42.5649, const_loss: 0.0007, l1_loss: 35.7862, fm_loss: 0.0057, perc_loss: 5.9320, edge: 0.1469
Epoch: [ 12], Batch: [ 100/ 404] | Total Time: 3h 5m 3s
d_loss: 1.3868, g_loss: 42.4824, const_loss: 0.0006, l1_loss: 35.5059, fm_loss: 0.0059, perc_loss: 6.1365, edge: 0.1401
Epoch: [ 12], Batch: [ 200/ 404] | Total Time: 3h 8m 48s
d_loss: 1.3868, g_loss: 42.3991, const_loss: 0.0008, l1_loss: 35.3835, fm_loss: 0.0060, perc_loss: 6.1578, edge: 0.1577
Epoch: [ 12], Batch: [ 300/ 404] | Total Time: 3h 12m 30s
d_loss: 1.3870, g_loss: 42.9327, const_loss: 0.0006, l1_loss: 35.9477, fm_loss: 0.0063, perc_loss: 6.1236, edge: 0.1611
Epoch: [ 12], Batch: [ 400/ 404] | Total Time: 3h 16m 13s
d_loss: 1.3868, g_loss: 45.0408, const_loss: 0.0008, l1_loss: 38.6932, fm_loss: 0.0062, perc_loss: 5.5066, edge: 0.1407
--- End of Epoch 12 --- Time: 901.8s ---
LR Scheduler stepped. Current LR G: 0.000160, LR D: 0.000160
Epoch: [ 13], Batch: [ 0/ 404] | Total Time: 3h 16m 21s
d_loss: 1.3867, g_loss: 43.2533, const_loss: 0.0007, l1_loss: 36.4425, fm_loss: 0.0055, perc_loss: 5.9704, edge: 0.1408
Epoch: [ 13], Batch: [ 100/ 404] | Total Time: 3h 20m 3s
d_loss: 1.3867, g_loss: 43.4132, const_loss: 0.0007, l1_loss: 36.6925, fm_loss: 0.0060, perc_loss: 5.8821, edge: 0.1386
Epoch: [ 13], Batch: [ 200/ 404] | Total Time: 3h 23m 46s
d_loss: 1.3868, g_loss: 43.2378, const_loss: 0.0006, l1_loss: 36.4400, fm_loss: 0.0056, perc_loss: 5.9638, edge: 0.1344
Epoch: [ 13], Batch: [ 300/ 404] | Total Time: 3h 27m 29s
d_loss: 1.3868, g_loss: 42.2146, const_loss: 0.0005, l1_loss: 35.3516, fm_loss: 0.0053, perc_loss: 5.9967, edge: 0.1671
Epoch: [ 13], Batch: [ 400/ 404] | Total Time: 3h 31m 12s
d_loss: 1.3867, g_loss: 42.4959, const_loss: 0.0006, l1_loss: 35.6885, fm_loss: 0.0054, perc_loss: 5.9505, edge: 0.1575
--- End of Epoch 13 --- Time: 899.7s ---
LR Scheduler stepped. Current LR G: 0.000153, LR D: 0.000153
Epoch: [ 14], Batch: [ 0/ 404] | Total Time: 3h 31m 20s
d_loss: 1.3873, g_loss: 41.2905, const_loss: 0.0007, l1_loss: 33.9039, fm_loss: 0.0055, perc_loss: 6.5157, edge: 0.1715
Epoch: [ 14], Batch: [ 100/ 404] | Total Time: 3h 35m 9s
d_loss: 1.3868, g_loss: 43.2348, const_loss: 0.0005, l1_loss: 36.7737, fm_loss: 0.0052, perc_loss: 5.6347, edge: 0.1274
Epoch: [ 14], Batch: [ 200/ 404] | Total Time: 3h 38m 52s
d_loss: 1.3868, g_loss: 43.8360, const_loss: 0.0006, l1_loss: 37.1701, fm_loss: 0.0054, perc_loss: 5.8181, edge: 0.1484
Epoch: [ 14], Batch: [ 300/ 404] | Total Time: 3h 42m 36s
d_loss: 1.3872, g_loss: 42.4426, const_loss: 0.0006, l1_loss: 36.0312, fm_loss: 0.0055, perc_loss: 5.5594, edge: 0.1526
Epoch: [ 14], Batch: [ 400/ 404] | Total Time: 3h 46m 19s
d_loss: 1.3869, g_loss: 42.4592, const_loss: 0.0007, l1_loss: 35.7291, fm_loss: 0.0057, perc_loss: 5.8944, edge: 0.1359
--- End of Epoch 14 --- Time: 906.5s ---
LR Scheduler stepped. Current LR G: 0.000145, LR D: 0.000145
Epoch: [ 15], Batch: [ 0/ 404] | Total Time: 3h 46m 27s
d_loss: 1.3867, g_loss: 44.9352, const_loss: 0.0018, l1_loss: 38.5217, fm_loss: 0.0060, perc_loss: 5.5911, edge: 0.1213
from 4370 to 4372
Model 4370 loaded successfully
unpickled total 8072 examples
Starting training from epoch 0/39...
Epoch: [ 0], Batch: [ 0/ 404] | Total Time: 4s
d_loss: 1.3868, g_loss: 42.4218, const_loss: 0.0007, l1_loss: 35.7491, fm_loss: 0.0056, perc_loss: 5.8285, edge: 0.1445
Checkpoint step 100 reached, but saving starts after step 200.
Epoch: [ 0], Batch: [ 100/ 404] | Total Time: 3m 44s
d_loss: 1.3868, g_loss: 43.8917, const_loss: 0.0007, l1_loss: 37.0345, fm_loss: 0.0053, perc_loss: 6.0298, edge: 0.1280
Epoch: [ 0], Batch: [ 200/ 404] | Total Time: 7m 30s
d_loss: 1.3867, g_loss: 42.2385, const_loss: 0.0005, l1_loss: 35.2022, fm_loss: 0.0057, perc_loss: 6.1645, edge: 0.1722
Epoch: [ 0], Batch: [ 300/ 404] | Total Time: 11m 14s
d_loss: 1.3868, g_loss: 43.2588, const_loss: 0.0005, l1_loss: 36.3550, fm_loss: 0.0058, perc_loss: 6.0860, edge: 0.1181
Epoch: [ 0], Batch: [ 400/ 404] | Total Time: 14m 59s
d_loss: 1.3868, g_loss: 43.4255, const_loss: 0.0006, l1_loss: 36.3434, fm_loss: 0.0056, perc_loss: 6.2237, edge: 0.1589
--- End of Epoch 0 --- Time: 905.5s ---
LR Scheduler stepped. Current LR G: 0.000203, LR D: 0.000203
Epoch: [ 1], Batch: [ 0/ 404] | Total Time: 15m 7s
d_loss: 1.3867, g_loss: 43.2473, const_loss: 0.0007, l1_loss: 36.3083, fm_loss: 0.0058, perc_loss: 6.0728, edge: 0.1664
Epoch: [ 1], Batch: [ 100/ 404] | Total Time: 18m 54s
d_loss: 1.3867, g_loss: 43.9066, const_loss: 0.0012, l1_loss: 36.4234, fm_loss: 0.0066, perc_loss: 6.6221, edge: 0.1600
Epoch: [ 1], Batch: [ 200/ 404] | Total Time: 22m 39s
d_loss: 1.3872, g_loss: 44.8959, const_loss: 0.0006, l1_loss: 38.6223, fm_loss: 0.0068, perc_loss: 5.4468, edge: 0.1261
Epoch: [ 1], Batch: [ 300/ 404] | Total Time: 26m 23s
d_loss: 1.3867, g_loss: 41.5403, const_loss: 0.0008, l1_loss: 34.7755, fm_loss: 0.0054, perc_loss: 5.9300, edge: 0.1353
Epoch: [ 1], Batch: [ 400/ 404] | Total Time: 30m 7s
d_loss: 1.3868, g_loss: 43.4487, const_loss: 0.0008, l1_loss: 36.6470, fm_loss: 0.0059, perc_loss: 5.9451, edge: 0.1565
--- End of Epoch 1 --- Time: 907.9s ---
LR Scheduler stepped. Current LR G: 0.000202, LR D: 0.000202
Epoch: [ 2], Batch: [ 0/ 404] | Total Time: 30m 15s
d_loss: 1.3867, g_loss: 44.3543, const_loss: 0.0006, l1_loss: 37.7874, fm_loss: 0.0058, perc_loss: 5.7290, edge: 0.1381
Epoch: [ 2], Batch: [ 100/ 404] | Total Time: 34m 5s
d_loss: 1.3868, g_loss: 42.5805, const_loss: 0.0009, l1_loss: 35.5037, fm_loss: 0.0055, perc_loss: 6.2288, edge: 0.1482
Epoch: [ 2], Batch: [ 200/ 404] | Total Time: 37m 50s
d_loss: 1.3871, g_loss: 43.7092, const_loss: 0.0008, l1_loss: 37.4240, fm_loss: 0.0051, perc_loss: 5.4465, edge: 0.1394
Epoch: [ 2], Batch: [ 300/ 404] | Total Time: 41m 35s
d_loss: 1.3868, g_loss: 43.0284, const_loss: 0.0006, l1_loss: 35.7864, fm_loss: 0.0055, perc_loss: 6.3605, edge: 0.1819
Epoch: [ 2], Batch: [ 400/ 404] | Total Time: 45m 25s
d_loss: 1.3868, g_loss: 42.4989, const_loss: 0.0006, l1_loss: 35.2576, fm_loss: 0.0053, perc_loss: 6.3773, edge: 0.1649
--- End of Epoch 2 --- Time: 917.6s ---
LR Scheduler stepped. Current LR G: 0.000200, LR D: 0.000200
Epoch: [ 3], Batch: [ 0/ 404] | Total Time: 45m 33s
d_loss: 1.3873, g_loss: 44.5468, const_loss: 0.0010, l1_loss: 38.0998, fm_loss: 0.0053, perc_loss: 5.6031, edge: 0.1442
Epoch: [ 3], Batch: [ 100/ 404] | Total Time: 49m 22s
d_loss: 1.3867, g_loss: 44.6867, const_loss: 0.0009, l1_loss: 38.0354, fm_loss: 0.0056, perc_loss: 5.8162, edge: 0.1353
Epoch: [ 3], Batch: [ 200/ 404] | Total Time: 53m 12s
d_loss: 1.3868, g_loss: 43.1432, const_loss: 0.0007, l1_loss: 35.9023, fm_loss: 0.0054, perc_loss: 6.4066, edge: 0.1347
Epoch: [ 3], Batch: [ 300/ 404] | Total Time: 57m 2s
d_loss: 1.3868, g_loss: 43.1446, const_loss: 0.0008, l1_loss: 36.3469, fm_loss: 0.0055, perc_loss: 5.9521, edge: 0.1460
Epoch: [ 3], Batch: [ 400/ 404] | Total Time: 1h 47s
d_loss: 1.3867, g_loss: 42.7332, const_loss: 0.0008, l1_loss: 35.9440, fm_loss: 0.0057, perc_loss: 5.9173, edge: 0.1721
--- End of Epoch 3 --- Time: 922.7s ---
LR Scheduler stepped. Current LR G: 0.000198, LR D: 0.000198
Epoch: [ 4], Batch: [ 0/ 404] | Total Time: 1h 55s
d_loss: 1.3867, g_loss: 42.5654, const_loss: 0.0007, l1_loss: 35.6864, fm_loss: 0.0053, perc_loss: 6.0142, edge: 0.1655
Epoch: [ 4], Batch: [ 100/ 404] | Total Time: 1h 4m 40s
d_loss: 1.3867, g_loss: 43.1918, const_loss: 0.0006, l1_loss: 36.7600, fm_loss: 0.0057, perc_loss: 5.5936, edge: 0.1386
Epoch: [ 4], Batch: [ 200/ 404] | Total Time: 1h 8m 30s
d_loss: 1.3867, g_loss: 43.3474, const_loss: 0.0007, l1_loss: 36.2357, fm_loss: 0.0056, perc_loss: 6.2403, edge: 0.1717
Epoch: [ 4], Batch: [ 300/ 404] | Total Time: 1h 12m 14s
d_loss: 1.3868, g_loss: 43.9761, const_loss: 0.0007, l1_loss: 37.3326, fm_loss: 0.0054, perc_loss: 5.8190, edge: 0.1251
Epoch: [ 4], Batch: [ 400/ 404] | Total Time: 1h 16m 3s
d_loss: 1.3871, g_loss: 42.6231, const_loss: 0.0007, l1_loss: 36.0737, fm_loss: 0.0058, perc_loss: 5.6988, edge: 0.1507
--- End of Epoch 4 --- Time: 916.0s ---
LR Scheduler stepped. Current LR G: 0.000195, LR D: 0.000195
Epoch: [ 5], Batch: [ 0/ 404] | Total Time: 1h 16m 11s
d_loss: 1.3869, g_loss: 42.6977, const_loss: 0.0007, l1_loss: 35.5844, fm_loss: 0.0060, perc_loss: 6.2494, edge: 0.1639
Epoch: [ 5], Batch: [ 100/ 404] | Total Time: 1h 19m 56s
d_loss: 1.3870, g_loss: 45.2015, const_loss: 0.0007, l1_loss: 38.9270, fm_loss: 0.0062, perc_loss: 5.4462, edge: 0.1280
Epoch: [ 5], Batch: [ 200/ 404] | Total Time: 1h 23m 45s
d_loss: 1.3868, g_loss: 43.8662, const_loss: 0.0006, l1_loss: 37.4397, fm_loss: 0.0054, perc_loss: 5.6070, edge: 0.1200
Epoch: [ 5], Batch: [ 300/ 404] | Total Time: 1h 27m 33s
d_loss: 1.3867, g_loss: 44.5590, const_loss: 0.0007, l1_loss: 37.9141, fm_loss: 0.0056, perc_loss: 5.8105, edge: 0.1348
Epoch: [ 5], Batch: [ 400/ 404] | Total Time: 1h 31m 23s
d_loss: 1.3868, g_loss: 42.8895, const_loss: 0.0006, l1_loss: 35.7821, fm_loss: 0.0056, perc_loss: 6.2478, edge: 0.1601
--- End of Epoch 5 --- Time: 920.0s ---
LR Scheduler stepped. Current LR G: 0.000192, LR D: 0.000192
Epoch: [ 6], Batch: [ 0/ 404] | Total Time: 1h 31m 31s
d_loss: 1.3868, g_loss: 44.1931, const_loss: 0.0007, l1_loss: 37.6833, fm_loss: 0.0058, perc_loss: 5.6839, edge: 0.1260
Epoch: [ 6], Batch: [ 100/ 404] | Total Time: 1h 35m 16s
d_loss: 1.3867, g_loss: 43.2407, const_loss: 0.0007, l1_loss: 36.6664, fm_loss: 0.0055, perc_loss: 5.7288, edge: 0.1459
Epoch: [ 6], Batch: [ 200/ 404] | Total Time: 1h 39m 0s
d_loss: 1.3867, g_loss: 43.0495, const_loss: 0.0007, l1_loss: 36.5305, fm_loss: 0.0054, perc_loss: 5.6820, edge: 0.1376
Epoch: [ 6], Batch: [ 300/ 404] | Total Time: 1h 42m 51s
d_loss: 1.3868, g_loss: 42.4365, const_loss: 0.0008, l1_loss: 35.6204, fm_loss: 0.0051, perc_loss: 5.9642, edge: 0.1527
Epoch: [ 6], Batch: [ 400/ 404] | Total Time: 1h 46m 39s
d_loss: 1.3867, g_loss: 44.0761, const_loss: 0.0007, l1_loss: 36.3202, fm_loss: 0.0053, perc_loss: 6.9017, edge: 0.1548
--- End of Epoch 6 --- Time: 915.3s ---
LR Scheduler stepped. Current LR G: 0.000188, LR D: 0.000188
Epoch: [ 7], Batch: [ 0/ 404] | Total Time: 1h 46m 47s
d_loss: 1.3868, g_loss: 42.4840, const_loss: 0.0009, l1_loss: 35.3169, fm_loss: 0.0053, perc_loss: 6.3293, edge: 0.1383
Epoch: [ 7], Batch: [ 100/ 404] | Total Time: 1h 50m 31s
d_loss: 1.3867, g_loss: 42.9597, const_loss: 0.0007, l1_loss: 36.3585, fm_loss: 0.0051, perc_loss: 5.7436, edge: 0.1585
Epoch: [ 7], Batch: [ 200/ 404] | Total Time: 1h 54m 22s
d_loss: 1.3869, g_loss: 44.0289, const_loss: 0.0009, l1_loss: 37.5598, fm_loss: 0.0052, perc_loss: 5.6291, edge: 0.1406
Epoch: [ 7], Batch: [ 300/ 404] | Total Time: 1h 58m 6s
d_loss: 1.3868, g_loss: 42.4456, const_loss: 0.0007, l1_loss: 35.4777, fm_loss: 0.0053, perc_loss: 6.1049, edge: 0.1638
Epoch: [ 7], Batch: [ 400/ 404] | Total Time: 2h 1m 51s
d_loss: 1.3872, g_loss: 42.3257, const_loss: 0.0006, l1_loss: 35.8204, fm_loss: 0.0051, perc_loss: 5.6772, edge: 0.1290
--- End of Epoch 7 --- Time: 911.8s ---
LR Scheduler stepped. Current LR G: 0.000184, LR D: 0.000184
Epoch: [ 8], Batch: [ 0/ 404] | Total Time: 2h 1m 59s
d_loss: 1.3867, g_loss: 43.3321, const_loss: 0.0006, l1_loss: 36.6211, fm_loss: 0.0052, perc_loss: 5.8523, edge: 0.1594
Epoch: [ 8], Batch: [ 100/ 404] | Total Time: 2h 5m 43s
d_loss: 1.3870, g_loss: 43.6134, const_loss: 0.0008, l1_loss: 36.9798, fm_loss: 0.0055, perc_loss: 5.8071, edge: 0.1268
Epoch: [ 8], Batch: [ 200/ 404] | Total Time: 2h 9m 33s
d_loss: 1.3867, g_loss: 42.5074, const_loss: 0.0006, l1_loss: 35.5562, fm_loss: 0.0052, perc_loss: 6.1058, edge: 0.1462
Epoch: [ 8], Batch: [ 300/ 404] | Total Time: 2h 13m 18s
d_loss: 1.3869, g_loss: 45.2836, const_loss: 0.0006, l1_loss: 38.9072, fm_loss: 0.0063, perc_loss: 5.5253, edge: 0.1509
Epoch: [ 8], Batch: [ 400/ 404] | Total Time: 2h 17m 6s
d_loss: 1.3868, g_loss: 42.8636, const_loss: 0.0006, l1_loss: 35.9238, fm_loss: 0.0055, perc_loss: 6.0865, edge: 0.1538
--- End of Epoch 8 --- Time: 916.0s ---
LR Scheduler stepped. Current LR G: 0.000179, LR D: 0.000179
Epoch: [ 9], Batch: [ 0/ 404] | Total Time: 2h 17m 15s
d_loss: 1.3868, g_loss: 42.7271, const_loss: 0.0006, l1_loss: 36.3519, fm_loss: 0.0053, perc_loss: 5.5638, edge: 0.1121
Epoch: [ 9], Batch: [ 100/ 404] | Total Time: 2h 20m 59s
d_loss: 1.3870, g_loss: 41.7035, const_loss: 0.0009, l1_loss: 34.7784, fm_loss: 0.0055, perc_loss: 6.0940, edge: 0.1314
Epoch: [ 9], Batch: [ 200/ 404] | Total Time: 2h 24m 43s
d_loss: 1.3868, g_loss: 42.6601, const_loss: 0.0006, l1_loss: 35.8857, fm_loss: 0.0057, perc_loss: 5.9058, edge: 0.1689
Epoch: [ 9], Batch: [ 300/ 404] | Total Time: 2h 28m 27s
d_loss: 1.3869, g_loss: 43.1895, const_loss: 0.0007, l1_loss: 36.7142, fm_loss: 0.0051, perc_loss: 5.6409, edge: 0.1353
Epoch: [ 9], Batch: [ 400/ 404] | Total Time: 2h 32m 17s
d_loss: 1.3868, g_loss: 42.5334, const_loss: 0.0007, l1_loss: 35.9074, fm_loss: 0.0052, perc_loss: 5.7804, edge: 0.1463
--- End of Epoch 9 --- Time: 910.6s ---
LR Scheduler stepped. Current LR G: 0.000173, LR D: 0.000173
Epoch: [ 10], Batch: [ 0/ 404] | Total Time: 2h 32m 25s
d_loss: 1.3867, g_loss: 42.6109, const_loss: 0.0007, l1_loss: 35.6642, fm_loss: 0.0055, perc_loss: 6.0874, edge: 0.1597
Epoch: [ 10], Batch: [ 100/ 404] | Total Time: 2h 36m 10s
d_loss: 1.3867, g_loss: 43.8889, const_loss: 0.0007, l1_loss: 37.1438, fm_loss: 0.0052, perc_loss: 5.8842, edge: 0.1616
Epoch: [ 10], Batch: [ 200/ 404] | Total Time: 2h 40m 1s
d_loss: 1.3867, g_loss: 43.5840, const_loss: 0.0005, l1_loss: 36.7410, fm_loss: 0.0050, perc_loss: 5.9699, edge: 0.1742
Epoch: [ 10], Batch: [ 300/ 404] | Total Time: 2h 43m 45s
d_loss: 1.3868, g_loss: 42.6587, const_loss: 0.0006, l1_loss: 36.2198, fm_loss: 0.0053, perc_loss: 5.5850, edge: 0.1547
Epoch: [ 10], Batch: [ 400/ 404] | Total Time: 2h 47m 29s
d_loss: 1.3868, g_loss: 43.1579, const_loss: 0.0004, l1_loss: 36.7405, fm_loss: 0.0047, perc_loss: 5.5950, edge: 0.1239
--- End of Epoch 10 --- Time: 912.4s ---
LR Scheduler stepped. Current LR G: 0.000168, LR D: 0.000168
Epoch: [ 11], Batch: [ 0/ 404] | Total Time: 2h 47m 37s
d_loss: 1.3867, g_loss: 42.9776, const_loss: 0.0008, l1_loss: 36.1732, fm_loss: 0.0058, perc_loss: 5.9656, edge: 0.1388
Epoch: [ 11], Batch: [ 100/ 404] | Total Time: 2h 51m 22s
d_loss: 1.3868, g_loss: 41.9314, const_loss: 0.0006, l1_loss: 35.1820, fm_loss: 0.0050, perc_loss: 5.9156, edge: 0.1348
Epoch: [ 11], Batch: [ 200/ 404] | Total Time: 2h 55m 11s
d_loss: 1.3867, g_loss: 43.6880, const_loss: 0.0007, l1_loss: 37.6697, fm_loss: 0.0051, perc_loss: 5.1887, edge: 0.1304
Epoch: [ 11], Batch: [ 300/ 404] | Total Time: 2h 58m 55s
d_loss: 1.3867, g_loss: 43.9663, const_loss: 0.0008, l1_loss: 37.4356, fm_loss: 0.0057, perc_loss: 5.6939, edge: 0.1369
Epoch: [ 11], Batch: [ 400/ 404] | Total Time: 3h 2m 40s
d_loss: 1.3868, g_loss: 43.7680, const_loss: 0.0008, l1_loss: 37.0855, fm_loss: 0.0055, perc_loss: 5.8418, edge: 0.1411
--- End of Epoch 11 --- Time: 910.7s ---
LR Scheduler stepped. Current LR G: 0.000161, LR D: 0.000161
Epoch: [ 12], Batch: [ 0/ 404] | Total Time: 3h 2m 48s
d_loss: 1.3868, g_loss: 42.2143, const_loss: 0.0006, l1_loss: 35.6331, fm_loss: 0.0049, perc_loss: 5.7420, edge: 0.1404
Epoch: [ 12], Batch: [ 100/ 404] | Total Time: 3h 6m 33s
d_loss: 1.3867, g_loss: 41.5626, const_loss: 0.0005, l1_loss: 35.1258, fm_loss: 0.0052, perc_loss: 5.5838, edge: 0.1540
Epoch: [ 12], Batch: [ 200/ 404] | Total Time: 3h 10m 17s
d_loss: 1.3868, g_loss: 41.9324, const_loss: 0.0006, l1_loss: 35.3236, fm_loss: 0.0054, perc_loss: 5.7686, edge: 0.1409
Epoch: [ 12], Batch: [ 300/ 404] | Total Time: 3h 14m 7s
d_loss: 1.3872, g_loss: 42.1586, const_loss: 0.0007, l1_loss: 35.6291, fm_loss: 0.0054, perc_loss: 5.6876, edge: 0.1425
Epoch: [ 12], Batch: [ 400/ 404] | Total Time: 3h 17m 55s
d_loss: 1.3868, g_loss: 44.8551, const_loss: 0.0006, l1_loss: 38.7821, fm_loss: 0.0056, perc_loss: 5.2187, edge: 0.1547
--- End of Epoch 12 --- Time: 914.8s ---
LR Scheduler stepped. Current LR G: 0.000155, LR D: 0.000155
Epoch: [ 13], Batch: [ 0/ 404] | Total Time: 3h 18m 3s
d_loss: 1.3868, g_loss: 43.9093, const_loss: 0.0006, l1_loss: 36.9046, fm_loss: 0.0054, perc_loss: 6.1484, edge: 0.1569
Epoch: [ 13], Batch: [ 100/ 404] | Total Time: 3h 21m 47s
d_loss: 1.3867, g_loss: 43.8407, const_loss: 0.0006, l1_loss: 36.9014, fm_loss: 0.0057, perc_loss: 6.0565, edge: 0.1832
Epoch: [ 13], Batch: [ 200/ 404] | Total Time: 3h 25m 32s
d_loss: 1.3868, g_loss: 42.2098, const_loss: 0.0006, l1_loss: 36.0490, fm_loss: 0.0047, perc_loss: 5.3285, edge: 0.1336
Epoch: [ 13], Batch: [ 300/ 404] | Total Time: 3h 29m 25s
d_loss: 1.3867, g_loss: 41.5623, const_loss: 0.0005, l1_loss: 35.2104, fm_loss: 0.0047, perc_loss: 5.5085, edge: 0.1449
Epoch: [ 13], Batch: [ 400/ 404] | Total Time: 3h 33m 16s
d_loss: 1.3867, g_loss: 41.7664, const_loss: 0.0006, l1_loss: 35.4298, fm_loss: 0.0047, perc_loss: 5.4925, edge: 0.1455
--- End of Epoch 13 --- Time: 920.6s ---
LR Scheduler stepped. Current LR G: 0.000148, LR D: 0.000148
Epoch: [ 14], Batch: [ 0/ 404] | Total Time: 3h 33m 24s
d_loss: 1.3873, g_loss: 41.0286, const_loss: 0.0006, l1_loss: 33.9437, fm_loss: 0.0050, perc_loss: 6.2605, edge: 0.1254
Epoch: [ 14], Batch: [ 100/ 404] | Total Time: 3h 37m 12s
d_loss: 1.3868, g_loss: 42.7233, const_loss: 0.0005, l1_loss: 36.6267, fm_loss: 0.0045, perc_loss: 5.2639, edge: 0.1342
Epoch: [ 14], Batch: [ 200/ 404] | Total Time: 3h 40m 57s
d_loss: 1.3867, g_loss: 43.2606, const_loss: 0.0005, l1_loss: 36.9733, fm_loss: 0.0047, perc_loss: 5.4402, edge: 0.1486
Epoch: [ 14], Batch: [ 300/ 404] | Total Time: 3h 44m 42s
d_loss: 1.3870, g_loss: 41.7968, const_loss: 0.0006, l1_loss: 35.8557, fm_loss: 0.0049, perc_loss: 5.1113, edge: 0.1310
Epoch: [ 14], Batch: [ 400/ 404] | Total Time: 3h 48m 27s
d_loss: 1.3869, g_loss: 41.6596, const_loss: 0.0007, l1_loss: 35.4178, fm_loss: 0.0050, perc_loss: 5.4012, edge: 0.1417
--- End of Epoch 14 --- Time: 911.8s ---
LR Scheduler stepped. Current LR G: 0.000141, LR D: 0.000141
Epoch: [ 15], Batch: [ 0/ 404] | Total Time: 3h 48m 35s
d_loss: 1.3868, g_loss: 44.3007, const_loss: 0.0014, l1_loss: 38.3000, fm_loss: 0.0052, perc_loss: 5.1734, edge: 0.1273
Epoch: [ 15], Batch: [ 100/ 404] | Total Time: 3h 52m 20s
d_loss: 1.3868, g_loss: 43.5145, const_loss: 0.0006, l1_loss: 37.4294, fm_loss: 0.0052, perc_loss: 5.2590, edge: 0.1269
Epoch: [ 15], Batch: [ 200/ 404] | Total Time: 3h 56m 7s
d_loss: 1.3868, g_loss: 42.2751, const_loss: 0.0007, l1_loss: 35.5385, fm_loss: 0.0047, perc_loss: 5.8925, edge: 0.1454
Epoch: [ 15], Batch: [ 300/ 404] | Total Time: 3h 59m 52s
d_loss: 1.3868, g_loss: 42.9310, const_loss: 0.0007, l1_loss: 36.4730, fm_loss: 0.0050, perc_loss: 5.6424, edge: 0.1166
Epoch: [ 15], Batch: [ 400/ 404] | Total Time: 4h 3m 39s
d_loss: 1.3867, g_loss: 41.7680, const_loss: 0.0007, l1_loss: 35.4580, fm_loss: 0.0046, perc_loss: 5.4813, edge: 0.1300
--- End of Epoch 15 --- Time: 912.2s ---
LR Scheduler stepped. Current LR G: 0.000133, LR D: 0.000133
Epoch: [ 16], Batch: [ 0/ 404] | Total Time: 4h 3m 48s
d_loss: 1.3868, g_loss: 42.2826, const_loss: 0.0008, l1_loss: 35.8880, fm_loss: 0.0053, perc_loss: 5.5522, edge: 0.1429
Epoch: [ 16], Batch: [ 100/ 404] | Total Time: 4h 7m 33s
d_loss: 1.3867, g_loss: 42.8522, const_loss: 0.0006, l1_loss: 36.5494, fm_loss: 0.0052, perc_loss: 5.4529, edge: 0.1507
Epoch: [ 16], Batch: [ 200/ 404] | Total Time: 4h 11m 17s
d_loss: 1.3868, g_loss: 44.3583, const_loss: 0.0007, l1_loss: 37.8203, fm_loss: 0.0050, perc_loss: 5.6704, edge: 0.1685
Epoch: [ 16], Batch: [ 300/ 404] | Total Time: 4h 15m 2s
d_loss: 1.3868, g_loss: 42.6110, const_loss: 0.0008, l1_loss: 36.3756, fm_loss: 0.0050, perc_loss: 5.3990, edge: 0.1373
Epoch: [ 16], Batch: [ 400/ 404] | Total Time: 4h 18m 53s
d_loss: 1.3867, g_loss: 43.1780, const_loss: 0.0008, l1_loss: 36.5863, fm_loss: 0.0052, perc_loss: 5.7242, edge: 0.1681
--- End of Epoch 16 --- Time: 913.0s ---
LR Scheduler stepped. Current LR G: 0.000126, LR D: 0.000126
Epoch: [ 17], Batch: [ 0/ 404] | Total Time: 4h 19m 1s
d_loss: 1.3868, g_loss: 42.2377, const_loss: 0.0007, l1_loss: 35.9453, fm_loss: 0.0048, perc_loss: 5.4417, edge: 0.1518
Epoch: [ 17], Batch: [ 100/ 404] | Total Time: 4h 22m 46s
d_loss: 1.3868, g_loss: 41.5699, const_loss: 0.0007, l1_loss: 35.2099, fm_loss: 0.0049, perc_loss: 5.5108, edge: 0.1502
Epoch: [ 17], Batch: [ 200/ 404] | Total Time: 4h 26m 30s
d_loss: 1.3867, g_loss: 42.1488, const_loss: 0.0006, l1_loss: 36.0449, fm_loss: 0.0046, perc_loss: 5.2626, edge: 0.1427
Epoch: [ 17], Batch: [ 300/ 404] | Total Time: 4h 30m 15s
d_loss: 1.3867, g_loss: 42.3343, const_loss: 0.0006, l1_loss: 35.5545, fm_loss: 0.0049, perc_loss: 5.9373, edge: 0.1437
Epoch: [ 17], Batch: [ 400/ 404] | Total Time: 4h 34m 0s
d_loss: 1.3868, g_loss: 42.8496, const_loss: 0.0005, l1_loss: 36.6626, fm_loss: 0.0047, perc_loss: 5.3503, edge: 0.1382
--- End of Epoch 17 --- Time: 907.1s ---
LR Scheduler stepped. Current LR G: 0.000118, LR D: 0.000118
Epoch: [ 18], Batch: [ 0/ 404] | Total Time: 4h 34m 8s
d_loss: 1.3873, g_loss: 42.9099, const_loss: 0.0005, l1_loss: 36.4866, fm_loss: 0.0046, perc_loss: 5.5842, edge: 0.1406
Epoch: [ 18], Batch: [ 100/ 404] | Total Time: 4h 37m 52s
d_loss: 1.3871, g_loss: 43.0459, const_loss: 0.0004, l1_loss: 36.9600, fm_loss: 0.0048, perc_loss: 5.2472, edge: 0.1401
Epoch: [ 18], Batch: [ 200/ 404] | Total Time: 4h 41m 37s
d_loss: 1.3867, g_loss: 42.4964, const_loss: 0.0006, l1_loss: 36.4058, fm_loss: 0.0045, perc_loss: 5.2595, edge: 0.1327
Epoch: [ 18], Batch: [ 300/ 404] | Total Time: 4h 45m 33s
d_loss: 1.3867, g_loss: 41.7404, const_loss: 0.0005, l1_loss: 35.3821, fm_loss: 0.0048, perc_loss: 5.5149, edge: 0.1447
這次的訓練, 幾乎等於沒成果.
from 4372 to 4374, figure:
又是有訓練等於沒訓練的狀態.
log:
Model 4372 loaded successfully
unpickled total 8072 examples
Starting training from epoch 0/39...
Epoch: [ 0], Batch: [ 0/ 404] | Total Time: 3s
d_loss: 1.3867, g_loss: 42.3076, const_loss: 0.0007, l1_loss: 35.8240, fm_loss: 0.0049, perc_loss: 5.6334, edge: 0.1512
Checkpoint step 100 reached, but saving starts after step 200.
Epoch: [ 0], Batch: [ 100/ 404] | Total Time: 3m 45s
d_loss: 1.3868, g_loss: 43.1132, const_loss: 0.0006, l1_loss: 36.7818, fm_loss: 0.0045, perc_loss: 5.5147, edge: 0.1182
Epoch: [ 0], Batch: [ 200/ 404] | Total Time: 7m 35s
d_loss: 1.3867, g_loss: 41.5843, const_loss: 0.0005, l1_loss: 35.0497, fm_loss: 0.0047, perc_loss: 5.7030, edge: 0.1330
Epoch: [ 0], Batch: [ 300/ 404] | Total Time: 11m 22s
d_loss: 1.3869, g_loss: 42.4748, const_loss: 0.0005, l1_loss: 35.9838, fm_loss: 0.0047, perc_loss: 5.6412, edge: 0.1512
Epoch: [ 0], Batch: [ 400/ 404] | Total Time: 15m 5s
d_loss: 1.3867, g_loss: 42.3632, const_loss: 0.0005, l1_loss: 35.9818, fm_loss: 0.0045, perc_loss: 5.5453, edge: 0.1376
--- End of Epoch 0 --- Time: 911.7s ---
LR Scheduler stepped. Current LR G: 0.000195, LR D: 0.000195
Epoch: [ 1], Batch: [ 0/ 404] | Total Time: 15m 13s
d_loss: 1.3867, g_loss: 42.4650, const_loss: 0.0006, l1_loss: 36.0191, fm_loss: 0.0048, perc_loss: 5.5900, edge: 0.1571
Epoch: [ 1], Batch: [ 100/ 404] | Total Time: 18m 58s
d_loss: 1.3868, g_loss: 43.1767, const_loss: 0.0008, l1_loss: 36.0256, fm_loss: 0.0053, perc_loss: 6.2998, edge: 0.1519
Epoch: [ 1], Batch: [ 200/ 404] | Total Time: 22m 46s
d_loss: 1.3873, g_loss: 45.7163, const_loss: 0.0005, l1_loss: 39.0309, fm_loss: 0.0061, perc_loss: 5.8284, edge: 0.1571
Epoch: [ 1], Batch: [ 300/ 404] | Total Time: 26m 30s
d_loss: 1.3867, g_loss: 41.2891, const_loss: 0.0007, l1_loss: 34.6888, fm_loss: 0.0047, perc_loss: 5.7356, edge: 0.1661
Epoch: [ 1], Batch: [ 400/ 404] | Total Time: 30m 15s
d_loss: 1.3868, g_loss: 42.9358, const_loss: 0.0007, l1_loss: 36.4276, fm_loss: 0.0049, perc_loss: 5.6521, edge: 0.1571
--- End of Epoch 1 --- Time: 910.0s ---
LR Scheduler stepped. Current LR G: 0.000194, LR D: 0.000194
Epoch: [ 2], Batch: [ 0/ 404] | Total Time: 30m 23s
d_loss: 1.3868, g_loss: 44.7843, const_loss: 0.0006, l1_loss: 38.0749, fm_loss: 0.0053, perc_loss: 5.8562, edge: 0.1539
Epoch: [ 2], Batch: [ 100/ 404] | Total Time: 34m 11s
d_loss: 1.3868, g_loss: 42.0486, const_loss: 0.0007, l1_loss: 35.3336, fm_loss: 0.0049, perc_loss: 5.8808, edge: 0.1353
Epoch: [ 2], Batch: [ 200/ 404] | Total Time: 37m 56s
d_loss: 1.3869, g_loss: 43.3752, const_loss: 0.0007, l1_loss: 37.3223, fm_loss: 0.0044, perc_loss: 5.2322, edge: 0.1223
Epoch: [ 2], Batch: [ 300/ 404] | Total Time: 41m 39s
d_loss: 1.3869, g_loss: 41.9012, const_loss: 0.0005, l1_loss: 35.2587, fm_loss: 0.0045, perc_loss: 5.7810, edge: 0.1631
Epoch: [ 2], Batch: [ 400/ 404] | Total Time: 45m 23s
d_loss: 1.3869, g_loss: 41.6242, const_loss: 0.0005, l1_loss: 34.9781, fm_loss: 0.0045, perc_loss: 5.8105, edge: 0.1371
--- End of Epoch 2 --- Time: 907.6s ---
LR Scheduler stepped. Current LR G: 0.000192, LR D: 0.000192
Epoch: [ 3], Batch: [ 0/ 404] | Total Time: 45m 31s
d_loss: 1.3869, g_loss: 43.3266, const_loss: 0.0008, l1_loss: 37.5852, fm_loss: 0.0046, perc_loss: 4.9225, edge: 0.1202
Epoch: [ 3], Batch: [ 100/ 404] | Total Time: 49m 15s
d_loss: 1.3867, g_loss: 43.6821, const_loss: 0.0007, l1_loss: 37.5480, fm_loss: 0.0045, perc_loss: 5.2865, edge: 0.1490
Epoch: [ 3], Batch: [ 200/ 404] | Total Time: 52m 58s
d_loss: 1.3869, g_loss: 41.6467, const_loss: 0.0006, l1_loss: 35.3910, fm_loss: 0.0043, perc_loss: 5.4387, edge: 0.1188
Epoch: [ 3], Batch: [ 300/ 404] | Total Time: 56m 49s
d_loss: 1.3867, g_loss: 42.7636, const_loss: 0.0007, l1_loss: 36.2626, fm_loss: 0.0047, perc_loss: 5.6449, edge: 0.1573
Epoch: [ 3], Batch: [ 400/ 404] | Total Time: 1h 32s
d_loss: 1.3867, g_loss: 42.1466, const_loss: 0.0007, l1_loss: 35.7326, fm_loss: 0.0048, perc_loss: 5.5956, edge: 0.1195
--- End of Epoch 3 --- Time: 909.2s ---
LR Scheduler stepped. Current LR G: 0.000190, LR D: 0.000190
Epoch: [ 4], Batch: [ 0/ 404] | Total Time: 1h 40s
d_loss: 1.3867, g_loss: 42.0112, const_loss: 0.0007, l1_loss: 35.4901, fm_loss: 0.0044, perc_loss: 5.6897, edge: 0.1329
Epoch: [ 4], Batch: [ 100/ 404] | Total Time: 1h 4m 24s
d_loss: 1.3868, g_loss: 43.2262, const_loss: 0.0005, l1_loss: 36.7807, fm_loss: 0.0048, perc_loss: 5.6054, edge: 0.1414
Epoch: [ 4], Batch: [ 200/ 404] | Total Time: 1h 8m 7s
d_loss: 1.3867, g_loss: 42.2165, const_loss: 0.0006, l1_loss: 35.8595, fm_loss: 0.0045, perc_loss: 5.5298, edge: 0.1288
Epoch: [ 4], Batch: [ 300/ 404] | Total Time: 1h 11m 56s
d_loss: 1.3868, g_loss: 43.4608, const_loss: 0.0006, l1_loss: 37.0895, fm_loss: 0.0044, perc_loss: 5.5541, edge: 0.1188
Epoch: [ 4], Batch: [ 400/ 404] | Total Time: 1h 15m 44s
d_loss: 1.3868, g_loss: 42.5029, const_loss: 0.0006, l1_loss: 36.0169, fm_loss: 0.0049, perc_loss: 5.6303, edge: 0.1569
--- End of Epoch 4 --- Time: 912.1s ---
LR Scheduler stepped. Current LR G: 0.000188, LR D: 0.000188
Epoch: [ 5], Batch: [ 0/ 404] | Total Time: 1h 15m 52s
d_loss: 1.3869, g_loss: 42.5441, const_loss: 0.0006, l1_loss: 35.5682, fm_loss: 0.0048, perc_loss: 6.1081, edge: 0.1692
Epoch: [ 5], Batch: [ 100/ 404] | Total Time: 1h 19m 36s
d_loss: 1.3868, g_loss: 44.5846, const_loss: 0.0006, l1_loss: 38.6831, fm_loss: 0.0052, perc_loss: 5.0556, edge: 0.1467
Epoch: [ 5], Batch: [ 200/ 404] | Total Time: 1h 23m 19s
d_loss: 1.3867, g_loss: 42.9204, const_loss: 0.0006, l1_loss: 37.0557, fm_loss: 0.0043, perc_loss: 5.0374, edge: 0.1290
Epoch: [ 5], Batch: [ 300/ 404] | Total Time: 1h 27m 6s
d_loss: 1.3868, g_loss: 44.1783, const_loss: 0.0006, l1_loss: 37.8118, fm_loss: 0.0046, perc_loss: 5.5255, edge: 0.1425
Epoch: [ 5], Batch: [ 400/ 404] | Total Time: 1h 30m 55s
d_loss: 1.3868, g_loss: 42.7239, const_loss: 0.0005, l1_loss: 35.7082, fm_loss: 0.0046, perc_loss: 6.1533, edge: 0.1639
--- End of Epoch 5 --- Time: 910.9s ---
LR Scheduler stepped. Current LR G: 0.000184, LR D: 0.000184
Epoch: [ 6], Batch: [ 0/ 404] | Total Time: 1h 31m 3s
d_loss: 1.3868, g_loss: 44.0517, const_loss: 0.0006, l1_loss: 37.6774, fm_loss: 0.0049, perc_loss: 5.5221, edge: 0.1534
Epoch: [ 6], Batch: [ 100/ 404] | Total Time: 1h 34m 47s
d_loss: 1.3867, g_loss: 42.6383, const_loss: 0.0006, l1_loss: 36.3918, fm_loss: 0.0047, perc_loss: 5.4067, edge: 0.1411
Epoch: [ 6], Batch: [ 200/ 404] | Total Time: 1h 38m 36s
d_loss: 1.3868, g_loss: 42.9531, const_loss: 0.0006, l1_loss: 36.5903, fm_loss: 0.0046, perc_loss: 5.5561, edge: 0.1082
Epoch: [ 6], Batch: [ 300/ 404] | Total Time: 1h 42m 19s
d_loss: 1.3869, g_loss: 41.8802, const_loss: 0.0007, l1_loss: 35.3674, fm_loss: 0.0044, perc_loss: 5.6844, edge: 0.1300
Epoch: [ 6], Batch: [ 400/ 404] | Total Time: 1h 46m 3s
d_loss: 1.3867, g_loss: 42.5197, const_loss: 0.0007, l1_loss: 35.7162, fm_loss: 0.0046, perc_loss: 5.9686, edge: 0.1364
--- End of Epoch 6 --- Time: 908.2s ---
LR Scheduler stepped. Current LR G: 0.000181, LR D: 0.000181
Epoch: [ 7], Batch: [ 0/ 404] | Total Time: 1h 46m 11s
d_loss: 1.3867, g_loss: 41.9273, const_loss: 0.0007, l1_loss: 35.3113, fm_loss: 0.0049, perc_loss: 5.7645, edge: 0.1525
Epoch: [ 7], Batch: [ 100/ 404] | Total Time: 1h 49m 54s
d_loss: 1.3868, g_loss: 42.8042, const_loss: 0.0006, l1_loss: 36.3527, fm_loss: 0.0045, perc_loss: 5.6411, edge: 0.1119
Epoch: [ 7], Batch: [ 200/ 404] | Total Time: 1h 53m 45s
d_loss: 1.3868, g_loss: 44.0913, const_loss: 0.0008, l1_loss: 37.7324, fm_loss: 0.0046, perc_loss: 5.5447, edge: 0.1154
Epoch: [ 7], Batch: [ 300/ 404] | Total Time: 1h 57m 29s
d_loss: 1.3867, g_loss: 41.5445, const_loss: 0.0006, l1_loss: 35.0723, fm_loss: 0.0044, perc_loss: 5.6465, edge: 0.1273
Epoch: [ 7], Batch: [ 400/ 404] | Total Time: 2h 1m 12s
d_loss: 1.3870, g_loss: 42.0324, const_loss: 0.0005, l1_loss: 35.7638, fm_loss: 0.0045, perc_loss: 5.4518, edge: 0.1185
--- End of Epoch 7 --- Time: 908.6s ---
LR Scheduler stepped. Current LR G: 0.000176, LR D: 0.000176
Epoch: [ 8], Batch: [ 0/ 404] | Total Time: 2h 1m 20s
d_loss: 1.3867, g_loss: 43.0246, const_loss: 0.0005, l1_loss: 36.5538, fm_loss: 0.0046, perc_loss: 5.6162, edge: 0.1561
Epoch: [ 8], Batch: [ 100/ 404] | Total Time: 2h 5m 4s
d_loss: 1.3870, g_loss: 43.0724, const_loss: 0.0008, l1_loss: 36.7425, fm_loss: 0.0046, perc_loss: 5.5074, edge: 0.1238
Epoch: [ 8], Batch: [ 200/ 404] | Total Time: 2h 8m 52s
d_loss: 1.3867, g_loss: 41.7246, const_loss: 0.0007, l1_loss: 35.2417, fm_loss: 0.0044, perc_loss: 5.6328, edge: 0.1517
Epoch: [ 8], Batch: [ 300/ 404] | Total Time: 2h 12m 36s
d_loss: 1.3868, g_loss: 44.9819, const_loss: 0.0006, l1_loss: 38.8198, fm_loss: 0.0051, perc_loss: 5.3268, edge: 0.1361
Epoch: [ 8], Batch: [ 400/ 404] | Total Time: 2h 16m 25s
d_loss: 1.3867, g_loss: 42.7609, const_loss: 0.0006, l1_loss: 35.9640, fm_loss: 0.0047, perc_loss: 5.9699, edge: 0.1283
--- End of Epoch 8 --- Time: 912.6s ---
LR Scheduler stepped. Current LR G: 0.000172, LR D: 0.000172
Epoch: [ 9], Batch: [ 0/ 404] | Total Time: 2h 16m 33s
d_loss: 1.3868, g_loss: 43.6566, const_loss: 0.0007, l1_loss: 36.9172, fm_loss: 0.0049, perc_loss: 5.8902, edge: 0.1504
Epoch: [ 9], Batch: [ 100/ 404] | Total Time: 2h 20m 21s
d_loss: 1.3869, g_loss: 41.2685, const_loss: 0.0007, l1_loss: 34.6136, fm_loss: 0.0050, perc_loss: 5.8144, edge: 0.1414
Epoch: [ 9], Batch: [ 200/ 404] | Total Time: 2h 24m 5s
d_loss: 1.3868, g_loss: 41.9175, const_loss: 0.0005, l1_loss: 35.5228, fm_loss: 0.0047, perc_loss: 5.5454, edge: 0.1508
Epoch: [ 9], Batch: [ 300/ 404] | Total Time: 2h 27m 48s
d_loss: 1.3869, g_loss: 43.3207, const_loss: 0.0006, l1_loss: 36.7237, fm_loss: 0.0047, perc_loss: 5.7548, edge: 0.1436
Epoch: [ 9], Batch: [ 400/ 404] | Total Time: 2h 31m 32s
d_loss: 1.3868, g_loss: 41.8050, const_loss: 0.0007, l1_loss: 35.6437, fm_loss: 0.0045, perc_loss: 5.3278, edge: 0.1349
--- End of Epoch 9 --- Time: 907.3s ---
LR Scheduler stepped. Current LR G: 0.000167, LR D: 0.000167
Epoch: [ 10], Batch: [ 0/ 404] | Total Time: 2h 31m 40s
d_loss: 1.3867, g_loss: 42.0562, const_loss: 0.0007, l1_loss: 35.5339, fm_loss: 0.0048, perc_loss: 5.6722, edge: 0.1512
Epoch: [ 10], Batch: [ 100/ 404] | Total Time: 2h 35m 24s
d_loss: 1.3867, g_loss: 42.8724, const_loss: 0.0007, l1_loss: 36.8670, fm_loss: 0.0044, perc_loss: 5.1734, edge: 0.1336
Epoch: [ 10], Batch: [ 200/ 404] | Total Time: 2h 39m 8s
d_loss: 1.3868, g_loss: 42.9490, const_loss: 0.0005, l1_loss: 36.4762, fm_loss: 0.0044, perc_loss: 5.5959, edge: 0.1787
Epoch: [ 10], Batch: [ 300/ 404] | Total Time: 2h 42m 51s
d_loss: 1.3868, g_loss: 42.0318, const_loss: 0.0006, l1_loss: 36.0271, fm_loss: 0.0046, perc_loss: 5.1652, edge: 0.1409
Epoch: [ 10], Batch: [ 400/ 404] | Total Time: 2h 46m 43s
d_loss: 1.3868, g_loss: 43.1819, const_loss: 0.0004, l1_loss: 36.7934, fm_loss: 0.0042, perc_loss: 5.5452, edge: 0.1453
--- End of Epoch 10 --- Time: 911.0s ---
LR Scheduler stepped. Current LR G: 0.000161, LR D: 0.000161
Epoch: [ 11], Batch: [ 0/ 404] | Total Time: 2h 46m 51s
d_loss: 1.3868, g_loss: 43.0789, const_loss: 0.0007, l1_loss: 36.2227, fm_loss: 0.0051, perc_loss: 6.0040, edge: 0.1530
Epoch: [ 11], Batch: [ 100/ 404] | Total Time: 2h 50m 36s
d_loss: 1.3867, g_loss: 41.3469, const_loss: 0.0006, l1_loss: 34.9844, fm_loss: 0.0044, perc_loss: 5.5195, edge: 0.1447
Epoch: [ 11], Batch: [ 200/ 404] | Total Time: 2h 54m 19s
d_loss: 1.3868, g_loss: 43.7426, const_loss: 0.0007, l1_loss: 37.7771, fm_loss: 0.0045, perc_loss: 5.1073, edge: 0.1596
Epoch: [ 11], Batch: [ 300/ 404] | Total Time: 2h 58m 4s
d_loss: 1.3867, g_loss: 43.5885, const_loss: 0.0007, l1_loss: 37.2148, fm_loss: 0.0048, perc_loss: 5.5307, edge: 0.1440
Epoch: [ 11], Batch: [ 400/ 404] | Total Time: 3h 1m 47s
d_loss: 1.3868, g_loss: 43.1139, const_loss: 0.0006, l1_loss: 36.8175, fm_loss: 0.0046, perc_loss: 5.4497, edge: 0.1482
--- End of Epoch 11 --- Time: 903.9s ---
LR Scheduler stepped. Current LR G: 0.000155, LR D: 0.000155
Epoch: [ 12], Batch: [ 0/ 404] | Total Time: 3h 1m 55s
d_loss: 1.3869, g_loss: 41.8963, const_loss: 0.0005, l1_loss: 35.5687, fm_loss: 0.0045, perc_loss: 5.4762, edge: 0.1531
Epoch: [ 12], Batch: [ 100/ 404] | Total Time: 3h 5m 38s
d_loss: 1.3867, g_loss: 41.5212, const_loss: 0.0004, l1_loss: 35.1668, fm_loss: 0.0047, perc_loss: 5.4971, edge: 0.1588
Epoch: [ 12], Batch: [ 200/ 404] | Total Time: 3h 9m 21s
d_loss: 1.3868, g_loss: 41.6233, const_loss: 0.0006, l1_loss: 35.1813, fm_loss: 0.0047, perc_loss: 5.5852, edge: 0.1581
Epoch: [ 12], Batch: [ 300/ 404] | Total Time: 3h 13m 4s
d_loss: 1.3871, g_loss: 41.8057, const_loss: 0.0006, l1_loss: 35.5503, fm_loss: 0.0048, perc_loss: 5.4117, edge: 0.1450
Epoch: [ 12], Batch: [ 400/ 404] | Total Time: 3h 16m 48s
d_loss: 1.3867, g_loss: 43.9011, const_loss: 0.0006, l1_loss: 38.3998, fm_loss: 0.0044, perc_loss: 4.6819, edge: 0.1210
--- End of Epoch 12 --- Time: 901.6s ---
LR Scheduler stepped. Current LR G: 0.000149, LR D: 0.000149
Epoch: [ 13], Batch: [ 0/ 404] | Total Time: 3h 16m 56s
d_loss: 1.3868, g_loss: 42.1832, const_loss: 0.0006, l1_loss: 36.1211, fm_loss: 0.0041, perc_loss: 5.2247, edge: 0.1393
Epoch: [ 13], Batch: [ 100/ 404] | Total Time: 3h 20m 40s
d_loss: 1.3867, g_loss: 42.6899, const_loss: 0.0005, l1_loss: 36.4057, fm_loss: 0.0046, perc_loss: 5.4240, edge: 0.1618
Epoch: [ 13], Batch: [ 200/ 404] | Total Time: 3h 24m 30s
d_loss: 1.3868, g_loss: 42.2721, const_loss: 0.0005, l1_loss: 36.1301, fm_loss: 0.0043, perc_loss: 5.2857, edge: 0.1582
Epoch: [ 13], Batch: [ 300/ 404] | Total Time: 3h 28m 15s
d_loss: 1.3867, g_loss: 42.0476, const_loss: 0.0004, l1_loss: 35.4340, fm_loss: 0.0043, perc_loss: 5.7763, edge: 0.1392
Epoch: [ 13], Batch: [ 400/ 404] | Total Time: 3h 31m 59s
d_loss: 1.3867, g_loss: 41.4526, const_loss: 0.0005, l1_loss: 35.3068, fm_loss: 0.0042, perc_loss: 5.3013, edge: 0.1465
--- End of Epoch 13 --- Time: 910.2s ---
LR Scheduler stepped. Current LR G: 0.000142, LR D: 0.000142
Epoch: [ 14], Batch: [ 0/ 404] | Total Time: 3h 32m 7s
d_loss: 1.3870, g_loss: 40.4208, const_loss: 0.0005, l1_loss: 33.6492, fm_loss: 0.0043, perc_loss: 5.9262, edge: 0.1473
Epoch: [ 14], Batch: [ 100/ 404] | Total Time: 3h 35m 50s
d_loss: 1.3868, g_loss: 42.4724, const_loss: 0.0005, l1_loss: 36.5005, fm_loss: 0.0039, perc_loss: 5.1549, edge: 0.1192
Epoch: [ 14], Batch: [ 200/ 404] | Total Time: 3h 39m 33s
d_loss: 1.3867, g_loss: 42.7400, const_loss: 0.0005, l1_loss: 36.7910, fm_loss: 0.0041, perc_loss: 5.1135, edge: 0.1376
Epoch: [ 14], Batch: [ 300/ 404] | Total Time: 3h 43m 16s
d_loss: 1.3871, g_loss: 41.4772, const_loss: 0.0005, l1_loss: 35.7374, fm_loss: 0.0043, perc_loss: 4.9197, edge: 0.1219
Epoch: [ 14], Batch: [ 400/ 404] | Total Time: 3h 47m 0s
d_loss: 1.3869, g_loss: 41.3557, const_loss: 0.0006, l1_loss: 35.3377, fm_loss: 0.0046, perc_loss: 5.1787, edge: 0.1409
--- End of Epoch 14 --- Time: 901.1s ---
LR Scheduler stepped. Current LR G: 0.000135, LR D: 0.000135
Epoch: [ 15], Batch: [ 0/ 404] | Total Time: 3h 47m 8s
d_loss: 1.3868, g_loss: 43.7652, const_loss: 0.0012, l1_loss: 38.1079, fm_loss: 0.0047, perc_loss: 4.8381, edge: 0.1199
Epoch: [ 15], Batch: [ 100/ 404] | Total Time: 3h 50m 53s
d_loss: 1.3868, g_loss: 42.7956, const_loss: 0.0005, l1_loss: 37.0885, fm_loss: 0.0044, perc_loss: 4.8823, edge: 0.1266
Epoch: [ 15], Batch: [ 200/ 404] | Total Time: 3h 54m 36s
d_loss: 1.3867, g_loss: 41.4111, const_loss: 0.0006, l1_loss: 35.2247, fm_loss: 0.0041, perc_loss: 5.3484, edge: 0.1399
Epoch: [ 15], Batch: [ 300/ 404] | Total Time: 3h 58m 19s
d_loss: 1.3868, g_loss: 42.3007, const_loss: 0.0006, l1_loss: 36.1378, fm_loss: 0.0042, perc_loss: 5.3188, edge: 0.1458
Epoch: [ 15], Batch: [ 400/ 404] | Total Time: 4h 2m 2s
d_loss: 1.3867, g_loss: 41.2812, const_loss: 0.0006, l1_loss: 35.2926, fm_loss: 0.0040, perc_loss: 5.1558, edge: 0.1347
--- End of Epoch 15 --- Time: 902.2s ---
LR Scheduler stepped. Current LR G: 0.000128, LR D: 0.000128
Epoch: [ 16], Batch: [ 0/ 404] | Total Time: 4h 2m 10s
d_loss: 1.3867, g_loss: 42.0856, const_loss: 0.0008, l1_loss: 35.8450, fm_loss: 0.0046, perc_loss: 5.4254, edge: 0.1164
Epoch: [ 16], Batch: [ 100/ 404] | Total Time: 4h 5m 53s
d_loss: 1.3867, g_loss: 42.2982, const_loss: 0.0005, l1_loss: 36.3583, fm_loss: 0.0045, perc_loss: 5.1040, edge: 0.1375
Epoch: [ 16], Batch: [ 200/ 404] | Total Time: 4h 9m 36s
d_loss: 1.3867, g_loss: 43.5005, const_loss: 0.0006, l1_loss: 37.4573, fm_loss: 0.0042, perc_loss: 5.2158, edge: 0.1292
Epoch: [ 16], Batch: [ 300/ 404] | Total Time: 4h 13m 19s
d_loss: 1.3868, g_loss: 42.1092, const_loss: 0.0007, l1_loss: 36.1724, fm_loss: 0.0044, perc_loss: 5.1064, edge: 0.1319
Epoch: [ 16], Batch: [ 400/ 404] | Total Time: 4h 17m 2s
d_loss: 1.3867, g_loss: 42.1752, const_loss: 0.0008, l1_loss: 36.1711, fm_loss: 0.0044, perc_loss: 5.1699, edge: 0.1357
--- End of Epoch 16 --- Time: 900.4s ---
LR Scheduler stepped. Current LR G: 0.000121, LR D: 0.000121
Epoch: [ 17], Batch: [ 0/ 404] | Total Time: 4h 17m 10s
d_loss: 1.3868, g_loss: 41.7871, const_loss: 0.0006, l1_loss: 35.8490, fm_loss: 0.0041, perc_loss: 5.1175, edge: 0.1226
Epoch: [ 17], Batch: [ 100/ 404] | Total Time: 4h 20m 53s
d_loss: 1.3868, g_loss: 41.1346, const_loss: 0.0006, l1_loss: 35.0387, fm_loss: 0.0044, perc_loss: 5.2456, edge: 0.1520
Epoch: [ 17], Batch: [ 200/ 404] | Total Time: 4h 24m 37s
d_loss: 1.3867, g_loss: 41.7004, const_loss: 0.0005, l1_loss: 35.8715, fm_loss: 0.0042, perc_loss: 4.9977, edge: 0.1331
Epoch: [ 17], Batch: [ 300/ 404] | Total Time: 4h 28m 20s
d_loss: 1.3867, g_loss: 41.6455, const_loss: 0.0006, l1_loss: 35.2612, fm_loss: 0.0042, perc_loss: 5.5470, edge: 0.1392
Epoch: [ 17], Batch: [ 400/ 404] | Total Time: 4h 32m 3s
d_loss: 1.3867, g_loss: 42.2203, const_loss: 0.0004, l1_loss: 36.4269, fm_loss: 0.0042, perc_loss: 4.9678, edge: 0.1277
--- End of Epoch 17 --- Time: 901.1s ---
LR Scheduler stepped. Current LR G: 0.000113, LR D: 0.000113
Epoch: [ 18], Batch: [ 0/ 404] | Total Time: 4h 32m 11s
d_loss: 1.3871, g_loss: 42.3854, const_loss: 0.0004, l1_loss: 36.2933, fm_loss: 0.0041, perc_loss: 5.2505, edge: 0.1437
Epoch: [ 18], Batch: [ 100/ 404] | Total Time: 4h 35m 55s
d_loss: 1.3870, g_loss: 42.5028, const_loss: 0.0004, l1_loss: 36.6866, fm_loss: 0.0042, perc_loss: 4.9710, edge: 0.1473
Epoch: [ 18], Batch: [ 200/ 404] | Total Time: 4h 39m 38s
d_loss: 1.3867, g_loss: 42.3294, const_loss: 0.0005, l1_loss: 36.3655, fm_loss: 0.0040, perc_loss: 5.1208, edge: 0.1452
Epoch: [ 18], Batch: [ 300/ 404] | Total Time: 4h 43m 22s
d_loss: 1.3867, g_loss: 41.1814, const_loss: 0.0005, l1_loss: 35.1467, fm_loss: 0.0043, perc_loss: 5.1981, edge: 0.1385