%pylab inline
from pylab import *
import codecs,string,os,sys,os.path,glob,re
Populating the interactive namespace from numpy and matplotlib
The CLSTM command line tools take their training data in HDF5 files (you will evenutally also be able to train directly from images saved on disk, as in ocropy, but that's not quite implemented yet). This illustrates how to store images into an HDF5 file and then how to run the clstmctc
training tool on the data.
We illustrate loading data with the uw3-500.tgz
dataset, available from tmbdev.net
.
!test -f uw3-500.tgz || wget -nd http://www.tmbdev.net/ocrdata/uw3-500.tgz
--2015-06-03 14:02:53-- http://www.tmbdev.net/ocrdata/uw3-500.tgz Resolving www.tmbdev.net (www.tmbdev.net)... 69.163.203.33 Connecting to www.tmbdev.net (www.tmbdev.net)|69.163.203.33|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1401847 (1.3M) [text/plain] Saving to: ‘uw3-500.tgz’ 100%[======================================>] 1,401,847 815KB/s in 1.7s 2015-06-03 14:02:55 (815 KB/s) - ‘uw3-500.tgz’ saved [1401847/1401847]
Let's untar the file unless it has already been untarred.
!test -d book || tar -zxvf uw3-500.tgz
book/ book/1241/ book/1241/010055.bin.png book/1241/010055.gt.txt book/0134/ book/0134/010016.gt.txt book/0134/010016.bin.png book/1180/ book/1180/010028.bin.png book/1180/010028.gt.txt book/0654/ book/0654/010043.bin.png book/0654/010043.gt.txt book/1583/ book/1583/010065.gt.txt book/1583/010065.bin.png book/1147/ book/1147/010030.bin.png book/1147/010030.gt.txt book/1395/ book/1395/010044.bin.png book/1395/010044.gt.txt book/0873/ book/0873/010074.bin.png book/0873/010074.gt.txt book/1480/ book/1480/010084.gt.txt book/1480/010084.bin.png book/0604/ book/0604/010010.bin.png book/0604/010010.gt.txt book/1074/ book/1074/010051.gt.txt book/1074/010040.gt.txt book/1074/010040.bin.png book/1074/010051.bin.png book/1437/ book/1437/010049.bin.png book/1437/010049.gt.txt book/0885/ book/0885/010006.gt.txt book/0885/010006.bin.png book/0214/ book/0214/010002.gt.txt book/0214/010002.bin.png book/0326/ book/0326/010061.gt.txt book/0326/010061.bin.png book/1377/ book/1377/010039.gt.txt book/1377/010039.bin.png book/0972/ book/0972/010026.gt.txt book/0972/010043.bin.png book/0972/010026.bin.png book/0972/010043.gt.txt book/0461/ book/0461/010074.bin.png book/0461/010074.gt.txt book/0157/ book/0157/010037.bin.png book/0157/010037.gt.txt book/0225/ book/0225/010074.bin.png book/0225/010074.gt.txt book/0395/ book/0395/010088.gt.txt book/0395/010063.bin.png book/0395/010063.gt.txt book/0395/010064.gt.txt book/0395/010001.bin.png book/0395/010001.gt.txt book/0395/010088.bin.png book/0395/010064.bin.png book/0074/ book/0074/010022.gt.txt book/0074/010022.bin.png book/1590/ book/1590/010011.bin.png book/1590/010011.gt.txt book/0659/ book/0659/010017.gt.txt book/0659/010017.bin.png book/0185/ book/0185/010089.bin.png book/0185/010089.gt.txt book/0835/ book/0835/010039.gt.txt book/0835/010039.bin.png book/1522/ book/1522/010058.gt.txt book/1522/010065.gt.txt book/1522/010065.bin.png book/1522/010058.bin.png book/0961/ book/0961/010054.gt.txt book/0961/010054.bin.png book/0599/ book/0599/010020.bin.png book/0599/010020.gt.txt book/0318/ book/0318/010085.bin.png book/0318/010085.gt.txt book/0931/ book/0931/010007.gt.txt book/0931/010007.bin.png book/0931/010023.gt.txt book/0931/010023.bin.png book/1520/ book/1520/010021.bin.png book/1520/010021.gt.txt book/0028/ book/0028/010012.bin.png book/0028/010004.gt.txt book/0028/010004.bin.png book/0028/010012.gt.txt book/1355/ book/1355/010089.bin.png book/1355/010089.gt.txt book/0460/ book/0460/010004.gt.txt book/0460/010004.bin.png book/1125/ book/1125/010013.gt.txt book/1125/010013.bin.png book/0539/ book/0539/010061.gt.txt book/0539/010061.bin.png book/0539/010091.gt.txt book/0539/010091.bin.png book/1488/ book/1488/010042.bin.png book/1488/010042.gt.txt book/1127/ book/1127/010102.gt.txt book/1127/010102.bin.png book/1336/ book/1336/010071.bin.png book/1336/010071.gt.txt book/1417/ book/1417/010035.gt.txt book/1417/010035.bin.png book/1305/ book/1305/010024.gt.txt book/1305/010024.bin.png book/0953/ book/0953/010015.gt.txt book/0953/010015.bin.png book/1031/ book/1031/010014.gt.txt book/1031/010014.bin.png book/0596/ book/0596/010002.gt.txt book/0596/010002.bin.png book/1174/ book/1174/010044.bin.png book/1174/010044.gt.txt book/1001/ book/1001/010061.gt.txt book/1001/010061.bin.png book/0355/ book/0355/010001.bin.png book/0355/010001.gt.txt book/0222/ book/0222/010007.gt.txt book/0222/010007.bin.png book/0222/010017.gt.txt book/0222/010017.bin.png book/1409/ book/1409/010004.gt.txt book/1409/010004.bin.png book/0066/ book/0066/010016.gt.txt book/0066/010016.bin.png book/0971/ book/0971/010005.bin.png book/0971/010005.gt.txt book/0191/ book/0191/010040.gt.txt book/0191/010040.bin.png book/1325/ book/1325/010036.bin.png book/1325/010036.gt.txt book/1544/ book/1544/010022.gt.txt book/1544/010022.bin.png book/0787/ book/0787/010001.bin.png book/0787/010001.gt.txt book/0331/ book/0331/010014.gt.txt book/0331/010014.bin.png book/0768/ book/0768/010009.gt.txt book/0768/010009.bin.png book/0911/ book/0911/010008.bin.png book/0911/010008.gt.txt book/0012/ book/0012/010036.bin.png book/0012/010036.gt.txt book/0012/010108.gt.txt book/0012/010108.bin.png book/0865/ book/0865/010022.gt.txt book/0865/010022.bin.png book/0865/010042.bin.png book/0865/010091.gt.txt book/0865/010042.gt.txt book/0865/010091.bin.png book/1552/ book/1552/010017.gt.txt book/1552/010017.bin.png book/1223/ book/1223/010060.bin.png book/1223/010057.bin.png book/1223/010060.gt.txt book/1223/010057.gt.txt book/1177/ book/1177/010046.bin.png book/1177/010046.gt.txt book/0857/ book/0857/010109.bin.png book/0857/010001.bin.png book/0857/010109.gt.txt book/0857/010001.gt.txt book/0705/ book/0705/010051.gt.txt book/0705/010045.gt.txt book/0705/010045.bin.png book/0705/010051.bin.png book/0808/ book/0808/010031.gt.txt book/0808/010031.bin.png book/0158/ book/0158/010007.gt.txt book/0158/010007.bin.png book/0468/ book/0468/010060.bin.png book/0468/010060.gt.txt book/0360/ book/0360/010027.bin.png book/0360/010027.gt.txt book/0540/ book/0540/010011.bin.png book/0540/010011.gt.txt book/0376/ book/0376/010048.gt.txt book/0376/010048.bin.png book/0376/010085.bin.png book/0376/010085.gt.txt book/0350/ book/0350/010054.gt.txt book/0350/010054.bin.png book/1415/ book/1415/010004.gt.txt book/1415/010004.bin.png book/1094/ book/1094/010075.gt.txt book/1094/010075.bin.png book/1090/ book/1090/010009.gt.txt book/1090/010009.bin.png book/1162/ book/1162/010004.gt.txt book/1162/010004.bin.png book/1276/ book/1276/010002.gt.txt book/1276/010002.bin.png book/0735/ book/0735/010002.gt.txt book/0735/010002.bin.png book/0183/ book/0183/010006.gt.txt book/0183/010006.bin.png book/0394/ book/0394/010060.bin.png book/0394/010060.gt.txt book/0864/ book/0864/010038.bin.png book/0864/010038.gt.txt book/0625/ book/0625/010007.gt.txt book/0625/010007.bin.png book/0913/ book/0913/010019.bin.png book/0913/010034.gt.txt book/0913/010034.bin.png book/0913/010019.gt.txt book/1462/ book/1462/010070.bin.png book/1462/010070.gt.txt book/0123/ book/0123/010003.gt.txt book/0123/010003.bin.png book/0123/010034.gt.txt book/0123/010034.bin.png book/1339/ book/1339/010054.gt.txt book/1339/010054.bin.png book/0989/ book/0989/010035.gt.txt book/0989/010035.bin.png book/1523/ book/1523/010039.gt.txt book/1523/010039.bin.png book/0712/ book/0712/010013.gt.txt book/0712/010013.bin.png book/1447/ book/1447/010091.gt.txt book/1447/010091.bin.png book/0373/ book/0373/010036.bin.png book/0373/010036.gt.txt book/0373/010046.bin.png book/0373/010001.bin.png book/0373/010001.gt.txt book/0373/010046.gt.txt book/1161/ book/1161/010021.bin.png book/1161/010021.gt.txt book/0005/ book/0005/010001.bin.png book/0005/010001.gt.txt book/0963/ book/0963/010026.gt.txt book/0963/010026.bin.png book/0476/ book/0476/010023.gt.txt book/0476/010023.bin.png book/1346/ book/1346/010012.bin.png book/1346/010012.gt.txt book/1423/ book/1423/010042.bin.png book/1423/010042.gt.txt book/0969/ book/0969/010098.bin.png book/0969/010098.gt.txt book/0969/010034.gt.txt book/0969/010034.bin.png book/0893/ book/0893/010116.bin.png book/0893/010116.gt.txt book/0291/ book/0291/010070.bin.png book/0291/010070.gt.txt book/1117/ book/1117/010049.bin.png book/1117/010049.gt.txt book/0207/ book/0207/010017.gt.txt book/0207/010017.bin.png book/1039/ book/1039/010051.gt.txt book/1039/010051.bin.png book/1240/ book/1240/010045.gt.txt book/1240/010045.bin.png book/1240/010038.bin.png book/1240/010038.gt.txt book/1021/ book/1021/010037.bin.png book/1021/010037.gt.txt book/1169/ book/1169/010033.gt.txt book/1169/010057.bin.png book/1169/010100.gt.txt book/1169/010057.gt.txt book/1169/010100.bin.png book/1169/010033.bin.png book/1185/ book/1185/010064.gt.txt book/1185/010064.bin.png book/0664/ book/0664/010033.gt.txt book/0664/010049.bin.png book/0664/010049.gt.txt book/0664/010033.bin.png book/0709/ book/0709/010002.gt.txt book/0709/010002.bin.png book/1109/ book/1109/010059.bin.png book/1109/010059.gt.txt book/1216/ book/1216/010035.gt.txt book/1216/010035.bin.png book/1139/ book/1139/010029.gt.txt book/1139/010077.gt.txt book/1139/010077.bin.png book/1139/010104.gt.txt book/1139/010029.bin.png book/1139/010104.bin.png book/0948/ book/0948/010014.gt.txt book/0948/010014.bin.png book/0995/ book/0995/010028.bin.png book/0995/010028.gt.txt book/1402/ book/1402/010020.bin.png book/1402/010020.gt.txt book/1323/ book/1323/010053.gt.txt book/1323/010053.bin.png book/0041/ book/0041/010039.gt.txt book/0041/010039.bin.png book/1473/ book/1473/010041.bin.png book/1473/010041.gt.txt book/0590/ book/0590/010033.gt.txt book/0590/010033.bin.png book/1002/ book/1002/010025.gt.txt book/1002/010025.bin.png book/0036/ book/0036/010043.bin.png book/0036/010043.gt.txt book/1307/ book/1307/010074.bin.png book/1307/010074.gt.txt book/1038/ book/1038/010005.bin.png book/1038/010005.gt.txt book/0639/ book/0639/010060.bin.png book/0639/010060.gt.txt book/1371/ book/1371/010031.gt.txt book/1371/010031.bin.png book/0349/ book/0349/010036.bin.png book/0349/010036.gt.txt book/1032/ book/1032/010071.bin.png book/1032/010071.gt.txt book/1394/ book/1394/010089.bin.png book/1394/010045.gt.txt book/1394/010089.gt.txt book/1394/010045.bin.png book/1492/ book/1492/010014.gt.txt book/1492/010014.bin.png book/1329/ book/1329/010049.bin.png book/1329/010049.gt.txt book/0895/ book/0895/010030.bin.png book/0895/010030.gt.txt book/0701/ book/0701/010023.gt.txt book/0701/010023.bin.png book/1443/ book/1443/010083.gt.txt book/1443/010072.gt.txt book/1443/010083.bin.png book/1443/010072.bin.png book/0629/ book/0629/010020.bin.png book/0629/010020.gt.txt book/1477/ book/1477/010082.bin.png book/1477/010082.gt.txt book/1477/010049.bin.png book/1477/010049.gt.txt book/1067/ book/1067/010058.gt.txt book/1067/010058.bin.png book/0630/ book/0630/010019.bin.png book/0630/010019.gt.txt book/0593/ book/0593/010078.bin.png book/0593/010078.gt.txt book/1549/ book/1549/010041.bin.png book/1549/010041.gt.txt book/1286/ book/1286/010007.gt.txt book/1286/010007.bin.png book/1286/010051.gt.txt book/1286/010051.bin.png book/1215/ book/1215/010013.gt.txt book/1215/010013.bin.png book/1150/ book/1150/010008.bin.png book/1150/010008.gt.txt book/0011/ book/0011/010111.gt.txt book/0011/010111.bin.png book/0795/ book/0795/010011.bin.png book/0795/010011.gt.txt book/0653/ book/0653/010086.bin.png book/0653/010086.gt.txt book/1252/ book/1252/010051.gt.txt book/1252/010051.bin.png book/1252/010028.bin.png book/1252/010028.gt.txt book/0422/ book/0422/010017.gt.txt book/0422/010017.bin.png book/1123/ book/1123/010102.gt.txt book/1123/010102.bin.png book/0833/ book/0833/010089.bin.png book/0833/010089.gt.txt book/0517/ book/0517/010039.gt.txt book/0517/010039.bin.png book/0315/ book/0315/010029.gt.txt book/0315/010029.bin.png book/0315/010034.gt.txt book/0315/010034.bin.png book/0877/ book/0877/010004.gt.txt book/0877/010004.bin.png book/0677/ book/0677/010015.gt.txt book/0677/010015.bin.png book/0095/ book/0095/010007.gt.txt book/0095/010007.bin.png book/0498/ book/0498/010019.bin.png book/0498/010019.gt.txt book/1376/ book/1376/010006.gt.txt book/1376/010006.bin.png book/0945/ book/0945/010008.bin.png book/0945/010008.gt.txt book/1352/ book/1352/010022.gt.txt book/1352/010022.bin.png book/1599/ book/1599/010019.bin.png book/1599/010019.gt.txt book/1420/ book/1420/010013.gt.txt book/1420/010059.bin.png book/1420/010059.gt.txt book/1420/010013.bin.png book/1004/ book/1004/010017.gt.txt book/1004/010017.bin.png book/0176/ book/0176/010077.gt.txt book/0176/010077.bin.png book/0410/ book/0410/010043.bin.png book/0410/010043.gt.txt book/1526/ book/1526/010034.gt.txt book/1526/010049.bin.png book/1526/010034.bin.png book/1526/010049.gt.txt book/1184/ book/1184/010045.gt.txt book/1184/010045.bin.png book/1398/ book/1398/010015.gt.txt book/1398/010015.bin.png book/0700/ book/0700/010021.bin.png book/0700/010021.gt.txt book/1141/ book/1141/010022.gt.txt book/1141/010022.bin.png book/1011/ book/1011/010053.gt.txt book/1011/010053.bin.png book/1442/ book/1442/010117.gt.txt book/1442/010117.bin.png book/0380/ book/0380/010074.bin.png book/0380/010074.gt.txt book/0661/ book/0661/010061.gt.txt book/0661/010061.bin.png book/0823/ book/0823/010034.gt.txt book/0823/010034.bin.png book/1422/ book/1422/010016.gt.txt book/1422/010016.bin.png book/0798/ book/0798/010045.gt.txt book/0798/010045.bin.png book/0896/ book/0896/010059.bin.png book/0896/010059.gt.txt book/1317/ book/1317/010023.gt.txt book/1317/010023.bin.png book/0097/ book/0097/010042.bin.png book/0097/010042.gt.txt book/1080/ book/1080/010090.gt.txt book/1080/010090.bin.png book/1140/ book/1140/010018.gt.txt book/1140/010018.bin.png book/0187/ book/0187/010046.bin.png book/0187/010046.gt.txt book/1263/ book/1263/010022.gt.txt book/1263/010018.gt.txt book/1263/010022.bin.png book/1263/010018.bin.png book/1483/ book/1483/010002.gt.txt book/1483/010002.bin.png book/0854/ book/0854/010034.gt.txt book/0854/010034.bin.png book/1244/ book/1244/010003.gt.txt book/1244/010003.bin.png book/0830/ book/0830/010130.gt.txt book/0830/010130.bin.png book/1524/ book/1524/010099.gt.txt book/1524/010099.bin.png book/0200/ book/0200/010007.gt.txt book/0200/010007.bin.png book/0934/ book/0934/010081.bin.png book/0934/010084.gt.txt book/0934/010081.gt.txt book/0934/010084.bin.png book/1569/ book/1569/010076.bin.png book/1569/010076.gt.txt book/1160/ book/1160/010026.gt.txt book/1160/010026.bin.png book/1199/ book/1199/010084.gt.txt book/1199/010084.bin.png book/1458/ book/1458/010008.bin.png book/1458/010008.gt.txt book/1557/ book/1557/010034.gt.txt book/1557/010034.bin.png book/1124/ book/1124/010041.bin.png book/1124/010041.gt.txt book/0192/ book/0192/010012.bin.png book/0192/010012.gt.txt book/0129/ book/0129/010016.gt.txt book/0129/010016.bin.png book/0635/ book/0635/010021.bin.png book/0635/010021.gt.txt book/1449/ book/1449/010014.gt.txt book/1449/010014.bin.png book/1449/010075.gt.txt book/1449/010051.gt.txt book/1449/010051.bin.png book/1449/010075.bin.png book/1425/ book/1425/010035.gt.txt book/1425/010035.bin.png book/1269/ book/1269/010064.gt.txt book/1269/010064.bin.png book/1075/ book/1075/010005.bin.png book/1075/010005.gt.txt book/1337/ book/1337/010044.bin.png book/1337/010044.gt.txt book/0643/ book/0643/010024.gt.txt book/0643/010024.bin.png book/0255/ book/0255/010031.gt.txt book/0255/010031.bin.png book/0623/ book/0623/010038.bin.png book/0623/010038.gt.txt book/1206/ book/1206/010018.gt.txt book/1206/010018.bin.png book/1069/ book/1069/010107.bin.png book/1069/010008.bin.png book/1069/010107.gt.txt book/1069/010008.gt.txt book/1548/ book/1548/010046.bin.png book/1548/010046.gt.txt book/1568/ book/1568/010075.gt.txt book/1568/010075.bin.png book/0496/ book/0496/010012.bin.png book/0496/010012.gt.txt book/0513/ book/0513/010064.gt.txt book/0513/010064.bin.png book/0878/ book/0878/010086.bin.png book/0878/010086.gt.txt book/0173/ book/0173/010009.gt.txt book/0173/010015.gt.txt book/0173/010009.bin.png book/0173/010015.bin.png book/1471/ book/1471/010085.bin.png book/1471/010085.gt.txt book/0796/ book/0796/010041.bin.png book/0796/010043.bin.png book/0796/010041.gt.txt book/0796/010043.gt.txt book/0859/ book/0859/010037.bin.png book/0859/010037.gt.txt book/1482/ book/1482/010042.bin.png book/1482/010042.gt.txt book/0955/ book/0955/010055.bin.png book/0955/010055.gt.txt book/1219/ book/1219/010024.gt.txt book/1219/010024.bin.png book/0412/ book/0412/010052.bin.png book/0412/010052.gt.txt book/1380/ book/1380/010003.gt.txt book/1380/010086.bin.png book/1380/010003.bin.png book/1380/010086.gt.txt book/1107/ book/1107/010018.gt.txt book/1107/010018.bin.png book/0194/ book/0194/010024.gt.txt book/0194/010024.bin.png book/0587/ book/0587/010001.bin.png book/0587/010001.gt.txt book/0587/010023.gt.txt book/0587/010023.bin.png book/0634/ book/0634/010085.bin.png book/0634/010085.gt.txt book/0528/ book/0528/010015.gt.txt book/0528/010015.bin.png book/0479/ book/0479/010101.bin.png book/0479/010101.gt.txt book/0175/ book/0175/010047.gt.txt book/0175/010047.bin.png book/0852/ book/0852/010006.gt.txt book/0852/010006.bin.png book/0937/ book/0937/010001.bin.png book/0937/010001.gt.txt book/0651/ book/0651/010072.gt.txt book/0651/010072.bin.png book/0651/010054.gt.txt book/0651/010054.bin.png book/0377/ book/0377/010039.gt.txt book/0377/010047.gt.txt book/0377/010044.bin.png book/0377/010047.bin.png book/0377/010044.gt.txt book/0377/010039.bin.png book/0956/ book/0956/010023.gt.txt book/0956/010023.bin.png book/1519/ book/1519/010046.bin.png book/1519/010046.gt.txt book/1260/ book/1260/010002.gt.txt book/1260/010002.bin.png book/0812/ book/0812/010008.bin.png book/0812/010008.gt.txt book/0638/ book/0638/010025.gt.txt book/0638/010025.bin.png book/1130/ book/1130/010061.gt.txt book/1130/010061.bin.png book/0240/ book/0240/010002.gt.txt book/0240/010027.bin.png book/0240/010027.gt.txt book/0240/010002.bin.png book/1508/ book/1508/010033.gt.txt book/1508/010033.bin.png book/0280/ book/0280/010022.gt.txt book/0280/010022.bin.png book/1126/ book/1126/010078.bin.png book/1126/010078.gt.txt book/0633/ book/0633/010014.gt.txt book/0633/010014.bin.png book/0570/ book/0570/010008.bin.png book/0570/010008.gt.txt book/0565/ book/0565/010001.bin.png book/0565/010001.gt.txt book/0814/ book/0814/010004.gt.txt book/0814/010004.bin.png book/1435/ book/1435/010035.gt.txt book/1435/010035.bin.png book/1318/ book/1318/010031.gt.txt book/1318/010031.bin.png book/1392/ book/1392/010041.bin.png book/1392/010041.gt.txt book/0142/ book/0142/010031.gt.txt book/0142/010031.bin.png book/0188/ book/0188/010059.bin.png book/0188/010059.gt.txt book/0414/ book/0414/010068.gt.txt book/0414/010068.bin.png book/0186/ book/0186/010028.bin.png book/0186/010028.gt.txt book/0272/ book/0272/010025.gt.txt book/0272/010008.bin.png book/0272/010025.bin.png book/0272/010008.gt.txt book/1132/ book/1132/010007.gt.txt book/1132/010007.bin.png book/1132/010098.bin.png book/1132/010098.gt.txt book/1132/010040.gt.txt book/1132/010040.bin.png book/1158/ book/1158/010035.gt.txt book/1158/010047.gt.txt book/1158/010015.gt.txt book/1158/010081.bin.png book/1158/010047.bin.png book/1158/010015.bin.png book/1158/010081.gt.txt book/1158/010035.bin.png book/1071/ book/1071/010112.gt.txt book/1071/010031.gt.txt book/1071/010031.bin.png book/1071/010061.gt.txt book/1071/010061.bin.png book/1071/010112.bin.png book/0407/ book/0407/010046.bin.png book/0407/010046.gt.txt book/0413/ book/0413/010076.bin.png book/0413/010009.gt.txt book/0413/010009.bin.png book/0413/010076.gt.txt book/0306/ book/0306/010003.gt.txt book/0306/010003.bin.png book/1138/ book/1138/010077.gt.txt book/1138/010077.bin.png book/0241/ book/0241/010015.gt.txt book/0241/010015.bin.png book/0603/ book/0603/010051.gt.txt book/0603/010031.gt.txt book/0603/010031.bin.png book/0603/010018.gt.txt book/0603/010051.bin.png book/0603/010018.bin.png book/0160/ book/0160/010036.bin.png book/0160/010036.gt.txt book/1502/ book/1502/010057.bin.png book/1502/010044.bin.png book/1502/010044.gt.txt book/1502/010057.gt.txt book/1406/ book/1406/010041.bin.png book/1406/010041.gt.txt book/1108/ book/1108/010026.gt.txt book/1108/010026.bin.png book/0508/ book/0508/010056.bin.png book/0508/010056.gt.txt book/0195/ book/0195/010019.bin.png book/0195/010019.gt.txt book/1357/ book/1357/010001.bin.png book/1357/010001.gt.txt book/0316/ book/0316/010063.bin.png book/0316/010063.gt.txt book/0488/ book/0488/010006.gt.txt book/0488/010006.bin.png book/0137/ book/0137/010045.gt.txt book/0137/010045.bin.png book/1591/ book/1591/010070.bin.png book/1591/010070.gt.txt book/1142/ book/1142/010118.gt.txt book/1142/010118.bin.png book/1452/ book/1452/010022.gt.txt book/1452/010022.bin.png book/0211/ book/0211/010012.bin.png book/0211/010012.gt.txt book/0575/ book/0575/010019.bin.png book/0575/010019.gt.txt book/1335/ book/1335/010027.bin.png book/1335/010027.gt.txt book/0855/ book/0855/010082.bin.png book/0855/010082.gt.txt book/0924/ book/0924/010019.bin.png book/0924/010019.gt.txt book/0640/ book/0640/010018.gt.txt book/0640/010018.bin.png book/1113/ book/1113/010043.bin.png book/1113/010043.gt.txt book/0189/ book/0189/010096.bin.png book/0189/010096.gt.txt book/1272/ book/1272/010076.bin.png book/1272/010077.gt.txt book/1272/010077.bin.png book/1272/010076.gt.txt book/0834/ book/0834/010025.gt.txt book/0834/010023.gt.txt book/0834/010025.bin.png book/0834/010023.bin.png book/0385/ book/0385/010009.gt.txt book/0385/010009.bin.png book/0456/ book/0456/010058.gt.txt book/0456/010058.bin.png book/1229/ book/1229/010003.gt.txt book/1229/010056.bin.png book/1229/010003.bin.png book/1229/010056.gt.txt book/1152/ book/1152/010058.gt.txt book/1152/010056.bin.png book/1152/010060.bin.png book/1152/010060.gt.txt book/1152/010056.gt.txt book/1152/010058.bin.png book/1469/ book/1469/010006.gt.txt book/1469/010008.bin.png book/1469/010006.bin.png book/1469/010008.gt.txt book/1262/ book/1262/010065.gt.txt book/1262/010065.bin.png book/1320/ book/1320/010046.bin.png book/1320/010046.gt.txt book/0626/ book/0626/010062.bin.png book/0626/010062.gt.txt book/0383/ book/0383/010001.bin.png book/0383/010001.gt.txt book/0628/ book/0628/010013.gt.txt book/0628/010013.bin.png book/1143/ book/1143/010021.bin.png book/1143/010021.gt.txt book/1459/ book/1459/010040.gt.txt book/1459/010040.bin.png book/0809/ book/0809/010039.gt.txt book/0809/010039.bin.png book/0489/ book/0489/010028.bin.png book/0489/010028.gt.txt book/1059/ book/1059/010098.bin.png book/1059/010098.gt.txt book/1531/ book/1531/010022.gt.txt book/1531/010022.bin.png book/0892/ book/0892/010031.gt.txt book/0892/010031.bin.png book/1026/ book/1026/010010.bin.png book/1026/010010.gt.txt book/1534/ book/1534/010061.gt.txt book/1534/010061.bin.png book/1466/ book/1466/010046.bin.png book/1466/010046.gt.txt book/0121/ book/0121/010017.gt.txt book/0121/010017.bin.png book/1061/ book/1061/010055.bin.png book/1061/010055.gt.txt book/0034/ book/0034/010034.gt.txt book/0034/010034.bin.png book/1546/ book/1546/010031.gt.txt book/1546/010031.bin.png book/0974/ book/0974/010042.bin.png book/0974/010042.gt.txt book/1375/ book/1375/010007.gt.txt book/1375/010007.bin.png book/0159/ book/0159/010007.gt.txt book/0159/010007.bin.png book/1490/ book/1490/010065.gt.txt book/1490/010065.bin.png book/1003/ book/1003/010049.bin.png book/1003/010049.gt.txt book/1171/ book/1171/010046.bin.png book/1171/010046.gt.txt book/1432/ book/1432/010005.bin.png book/1432/010005.gt.txt book/1077/ book/1077/010059.bin.png book/1077/010059.gt.txt book/1282/ book/1282/010021.bin.png book/1282/010021.gt.txt book/1051/ book/1051/010039.gt.txt book/1051/010039.bin.png book/0293/ book/0293/010123.gt.txt book/0293/010123.bin.png book/0523/ book/0523/010068.gt.txt book/0523/010068.bin.png book/1528/ book/1528/010010.bin.png book/1528/010010.gt.txt book/0853/ book/0853/010092.bin.png book/0853/010092.gt.txt book/1389/ book/1389/010003.gt.txt book/1389/010003.bin.png book/1387/ book/1387/010035.gt.txt book/1387/010064.gt.txt book/1387/010064.bin.png book/1387/010035.bin.png book/1078/ book/1078/010076.bin.png book/1078/010076.gt.txt book/1554/ book/1554/010025.gt.txt book/1554/010025.bin.png book/1164/ book/1164/010063.bin.png book/1164/010063.gt.txt book/1164/010084.gt.txt book/1164/010084.bin.png book/0313/ book/0313/010021.bin.png book/0313/010021.gt.txt book/1314/ book/1314/010062.bin.png book/1314/010062.gt.txt book/0106/ book/0106/010002.gt.txt book/0106/010002.bin.png book/1271/ book/1271/010003.gt.txt book/1271/010003.bin.png book/1478/ book/1478/010047.gt.txt book/1478/010047.bin.png book/0022/ book/0022/010005.bin.png book/0022/010005.gt.txt book/1183/ book/1183/010072.gt.txt book/1183/010072.bin.png book/1547/ book/1547/010006.gt.txt book/1547/010052.bin.png book/1547/010006.bin.png book/1547/010052.gt.txt book/1386/ book/1386/010032.bin.png book/1386/010032.gt.txt book/1489/ book/1489/010089.bin.png book/1489/010089.gt.txt book/1489/010091.gt.txt book/1489/010091.bin.png book/1193/ book/1193/010046.bin.png book/1193/010046.gt.txt book/0999/ book/0999/010003.gt.txt book/0999/010047.gt.txt book/0999/010003.bin.png book/0999/010034.gt.txt book/0999/010034.bin.png book/0999/010047.bin.png book/0999/010022.gt.txt book/0999/010050.bin.png book/0999/010022.bin.png book/0999/010050.gt.txt book/1319/ book/1319/010058.gt.txt book/1319/010058.bin.png book/0062/ book/0062/010033.gt.txt book/0062/010033.bin.png book/0525/ book/0525/010030.bin.png book/0525/010030.gt.txt book/1237/ book/1237/010059.bin.png book/1237/010059.gt.txt book/1453/ book/1453/010018.gt.txt book/1453/010018.bin.png book/0417/ book/0417/010033.gt.txt book/0417/010056.bin.png book/0417/010056.gt.txt book/0417/010033.bin.png book/0408/ book/0408/010003.gt.txt book/0408/010047.gt.txt book/0408/010003.bin.png book/0408/010049.bin.png book/0408/010047.bin.png book/0408/010049.gt.txt book/0026/ book/0026/010010.bin.png book/0026/010010.gt.txt book/1175/ book/1175/010006.gt.txt book/1175/010006.bin.png book/0941/ book/0941/010039.gt.txt book/0941/010039.bin.png book/0518/ book/0518/010044.bin.png book/0518/010044.gt.txt book/0370/ book/0370/010099.gt.txt book/0370/010099.bin.png book/0648/ book/0648/010028.bin.png book/0648/010028.gt.txt book/0140/ book/0140/010095.gt.txt book/0140/010095.bin.png book/1295/ book/1295/010009.gt.txt book/1295/010009.bin.png book/0755/ book/0755/010030.bin.png book/0755/010030.gt.txt book/0416/ book/0416/010055.bin.png book/0416/010055.gt.txt book/0601/ book/0601/010068.gt.txt book/0601/010068.bin.png book/0104/ book/0104/010009.gt.txt book/0104/010009.bin.png book/1507/ book/1507/010042.bin.png book/1507/010042.gt.txt book/1083/ book/1083/010056.bin.png book/1083/010056.gt.txt book/1116/ book/1116/010030.bin.png book/1116/010089.bin.png book/1116/010089.gt.txt book/1116/010030.gt.txt book/1137/ book/1137/010010.bin.png book/1137/010023.gt.txt book/1137/010023.bin.png book/1137/010010.gt.txt book/0868/ book/0868/010037.bin.png book/0868/010037.gt.txt book/0868/010028.bin.png book/0868/010028.gt.txt book/0033/ book/0033/010060.bin.png book/0033/010060.gt.txt book/0328/ book/0328/010050.bin.png book/0328/010050.gt.txt book/1155/ book/1155/010069.gt.txt book/1155/010069.bin.png book/0605/ book/0605/010026.gt.txt book/0605/010026.bin.png book/0526/ book/0526/010045.gt.txt book/0526/010045.bin.png book/0706/ book/0706/010004.gt.txt book/0706/010004.bin.png book/1064/ book/1064/010095.gt.txt book/1064/010095.bin.png book/0917/ book/0917/010054.gt.txt book/0917/010054.bin.png book/1348/ book/1348/010024.gt.txt book/1348/010024.bin.png book/0317/ book/0317/010093.gt.txt book/0317/010133.bin.png book/0317/010133.gt.txt book/0317/010093.bin.png book/0317/010042.bin.png book/0317/010042.gt.txt book/0591/ book/0591/010099.gt.txt book/0591/010099.bin.png book/1036/ book/1036/010021.bin.png book/1036/010021.gt.txt book/1539/ book/1539/010002.gt.txt book/1539/010002.bin.png book/0658/ book/0658/010002.gt.txt book/0658/010002.bin.png book/1133/ book/1133/010066.bin.png book/1133/010066.gt.txt book/1040/ book/1040/010056.bin.png book/1040/010056.gt.txt book/1253/ book/1253/010027.bin.png book/1253/010027.gt.txt book/0415/ book/0415/010035.gt.txt book/0415/010033.gt.txt book/0415/010057.bin.png book/0415/010057.gt.txt book/0415/010035.bin.png book/0415/010033.bin.png book/0749/ book/0749/010030.bin.png book/0749/010030.gt.txt book/1208/ book/1208/010032.bin.png book/1208/010032.gt.txt book/1208/010017.gt.txt book/1208/010017.bin.png book/0691/ book/0691/010073.bin.png book/0691/010073.gt.txt book/1243/ book/1243/010056.bin.png book/1243/010056.gt.txt book/1430/ book/1430/010025.gt.txt book/1430/010025.bin.png book/0172/ book/0172/010072.gt.txt book/0172/010038.bin.png book/0172/010038.gt.txt book/0172/010072.bin.png book/1362/ book/1362/010116.bin.png book/1362/010116.gt.txt book/1362/010055.bin.png book/1362/010055.gt.txt book/0184/ book/0184/010002.gt.txt book/0184/010002.bin.png book/1363/ book/1363/010007.gt.txt book/1363/010007.bin.png book/1418/ book/1418/010025.gt.txt book/1418/010052.bin.png book/1418/010045.gt.txt book/1418/010025.bin.png book/1418/010045.bin.png book/1418/010052.gt.txt book/0043/ book/0043/010006.gt.txt book/0043/010006.bin.png book/1170/ book/1170/010108.gt.txt book/1170/010108.bin.png book/1168/ book/1168/010023.gt.txt book/1168/010023.bin.png book/0174/ book/0174/010060.bin.png book/0174/010060.gt.txt book/0015/ book/0015/010023.gt.txt book/0015/010023.bin.png book/1070/ book/1070/010027.bin.png book/1070/010027.gt.txt book/0607/ book/0607/010008.bin.png book/0607/010008.gt.txt book/1367/ book/1367/010084.gt.txt book/1367/010084.bin.png book/0939/ book/0939/010086.bin.png book/0939/010086.gt.txt book/1097/ book/1097/010025.gt.txt book/1097/010025.bin.png book/0038/ book/0038/010048.gt.txt book/0038/010048.bin.png book/0785/ book/0785/010022.gt.txt book/0785/010022.bin.png book/1015/ book/1015/010015.gt.txt book/1015/010015.bin.png book/0257/ book/0257/010001.bin.png book/0257/010001.gt.txt book/0981/ book/0981/010043.bin.png book/0981/010043.gt.txt
The UW3-500 dataset is a collection of text line images and corresponding ground truth transcription. It's organized as a directory tree of the form book/<page_no>/<line_id>.bin.png
etc.
!ls book/0005/010001.*
book/0005/010001.bin.png book/0005/010001.gt.txt
Let's now run clstmctc
training. We report every 100 training steps. Since we didn't dewarp or size-normalize the lines, we need to use a dewarp=center
argument to training.
!dewarp=center report_every=500 save_name=test save_every=10000 ntrain=11000 ../clstmctc uw3-500.h5
params hg_version e8159ab3bd6d+ lrate 0.0001 nhidden 100 nhidden2 -1 pseudo_batch 1 momentum 0.9 type BIDILSTM LSTM SoftmaxLayer center_normalizer 4 1 0.3 dataset 500 48 center Segmentation fault (core dumped)
!ls book/*/*.bin.png | sort -r > uw3.files
!sed 100q uw3.files > uw3-test.files
!sed 1,100d uw3.files > uw3-train.files
!wc -l uw3*.files
100 uw3-test.files 400 uw3-train.files 500 uw3.files 1000 total
!params=1 save_name=uw3small save_every=1000 report_every=100 maxtrain=50000 test_every=1000 ../clstmocrtrain uw3-train.files uw3-test.files
#: seed = 0 got 400 files, 100 tests got 83 classes #: nhidden = 100 #: momentum = 0.9 #: rate = 0.0001 .stacked: 0.0001 0.9 in 0 48 out 0 83 .stacked.parallel: 0.0001 0.9 in 0 48 out 0 200 .stacked.parallel.lstm: 0.0001 0.9 in 0 48 out 0 100 .stacked.parallel.reversed: 0.0001 0.9 in 0 48 out 0 100 .stacked.parallel.reversed.lstm: 0.0001 0.9 in 0 48 out 0 100 .stacked.softmax: 0.0001 0.9 in 0 200 out 0 83 #: maxtrain = 50000 #: save_every = 1000 #: save_name = uw3small #: report_every = 100 #: display = 0 #: test_every = 1000 0 TRU Table 3. Percent responses to questions ask- ALN OUT 100 TRU that only those neighbors are taken into account which do not ALN ttatt eeeeer e nn n o ooot OUT 200 TRU rived it, and the equation follows: ALN rived it,, a d h eeeeeqqaiioon flllwss::: OUT 300 TRU illustrate typical scene content. In order to obtain ALN illstraatetyyypical scene cconnttnI. Innorder to obtain OUT 400 TRU Table 3. Percent responses to questions ask- ALN ale 3Percent resonsssess oqquestsnssk- OUT eeeeeeeeeeee eeeee 500 TRU sition between pre-assembly relations and post-assembly ALN sition betweenpre-ssemby rellatiions aand ppost-asssemlyy OUT a a a a a a 600 TRU increasing commitment to im- ALN inncreasing cooommmitmmeent to imm- OUT eeeeee eeeeeeeeeee eeeeee eee 700 TRU gineering from ISU in 1984. From 1977 to 1982, he was a ALN gineeering froom IU iin 19884. From197 to718, hehe wass a OUT aoees ao aeoao aa e a oao oe a aesao 800 TRU Fig. 2. (a) The energy spectrum of the neutron pulses emerging from any of the LANSCE moderators is very broad. Details of the spectrum vary ALN i. 2. The energyysspesscctrruuum of thee neeeutrroon ppulsses emmerginngg frooom aany of thehe LNCE deratortrs iss eryyy broad. Dttaiss of thee sspeectrum vary OUT a aaa aaeaaaaaeaaaaaaaaaeai aaaeaaaaaaaaeaalaeaaaaaiaaa aaaaaaeaaaaeai aa ata eaaaaaaaaa eeaa aaaai eaaaaaeai aaeaaaaaaaaaeaaa 900 TRU 306 ALN 3066 OUT ae ERROR 1000 0.62998 2862 4543 1000 TRU 2. The major additional assumption at this well site is that there are both ALN 2. The majoor adddditionnnal asssuumptionnn at this well ste i that there are boothh OUT toe oo eoe aelot eoooot oeeeoaoot oan ot ti et i e not teee oee oeottn 1100 TRU Our work to date has been focused on an initial set ALN Ouuur worrk to date has been foocused on ann initial et OUT ttnn oeoat oo dnte n ee ooeeneed on inn nnitiil et 1200 TRU 2755 ALN 2755 OUT sa 1300 TRU 306 ALN 306 OUT t 1400 TRU instead of a series over time, to obtain ALN innsteaaad of a eries over immme, to obtainn OUT innslesned l n eries oer innne eo oltsninn 1500 TRU is one in which the neural activity is generally in ALN is one in which te neural activity is generally nn OUT is orne rn etnietn he ernral aetieity s ernerally rn 1600 TRU at the creative aspects of design and at the intricate ALN at he reative aspects of design and at heintricate OUT at he neatipe aseots l dessn aod at heintitate 1700 TRU page). ALN pagage) OUT ge 1800 TRU increasing commitment to im- ALN increasing commitment o im- OUT ircreasitug oruritnert oir- 1900 TRU The data for N and H were obtained from the ALN The data for N and H were obtained fromm he OUT The data or y anod f yere htaineod ronn he ERROR 2000 0.287255 1305 4543 2000 TRU mother to the benefits of differen- ALN mother to the benefits of differen- OUT rother o he oreflts f llffrer- 2100 TRU Brittonia, 45(1), 1993, pp. 28-31. ALN BBirtonia, 45(1), 1993, pp. 28-31 OUT i2errroeina wlol yyp, oo, f-Sl 2200 TRU unit quaternions). ALN unit quatenions) OUT unit puatenions 2300 TRU Policymakers seek data to monitor ALN PPolicymakers seek data o monitor OUT T2olicynaiersseei data o nonitor 2400 TRU In recent years a number of papers have appeared in ALN In recent years a number ofpaapers have appeared in OUT lp ecen1 pevp w aumnher f oepers nave appearedin 2500 TRU processes of ripening and shedding in the ALN processes of ripening and shedding in tthe OUT processes of ipening and sheddingin hie 2600 TRU The slugs in the flow path and the source term at the repository are ALN The slugs in the flow path and the source term at the repository are OUT t2he slugsin he low path and the source ermn at he repositorg are 2700 TRU for the Mozambique Belt away from the Kenya ALN for the Mozambiqque Belt away from the Kenya OUT for he Moxambiouc Sclt away from the enya 2800 TRU from the image at the predicted point and performs cross-correlation ALN from the image at the predicted point and perfoms cross-correlation OUT fmnm theimage at the predicted oirt and perfomns moss-correlation 2900 TRU move a pointing device to the location of the ALN move a pointing device to the location of the OUT move a pointing device to thelocation of he ERROR 3000 0.125468 570 4543 3000 TRU burners.'' Further, a lighthouse could identify itself by ex- ALN burners.'' Further, a lighthouse could identify itself by ex- OUT burners urther, a ighthouse couldidentifyitself by er- 3100 TRU Chilopoda, and Araneae. Several pinned salamanders are also present. ALN Chilopoda, and Araneae. Several pinned salamanders are also present OUT Chilopoda. and hraneae. Several pinned salamanders are also present 3200 TRU 10%) relative to the much greater limitation on photo- ALN 10%%%) relative to the much greater limitation on photo- OUT 1 relative o the much greater limitation on photo- 3300 TRU IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 8, NO. 5, OCTOBER 1992 ALN IEEEEEE TTRANSACTIONS OON ROOBOTICS AND AUUTOMATION, OL. 8, NO. 5, OCTOBBERER 1992 OUT 1ESESES TTRANSAcT1oNS oN RCEsCTEcSs AahNb AtTToMA1CN, o1, s, wwo. s. octoEERER wt2 3400 TRU burners.'' Further, a lighthouse could identify itself by ex- ALN burners.' Further, a lighthouse could identify itself by ex- OUT burners Further, a iighthouse could identify itself y ex- 3500 TRU lengths of the straight regions. ALN lengths of the straight regions OUT lengths af the straight regions 3600 TRU called the Navlab 2. ALN called the Navlab 2 OUT called the Nawlab 2 3700 TRU tures for each frame. These features are dense, stable, ALN tures for each frame. These features are dense, stable, OUT tures for each frame, These features are dense, stalble 3800 TRU microstructures obtained under various cooling condi- ALN microstructures obtained under various cooling condi- OUT microstructures obtained under warious cooling condi- 3900 TRU for the Mozambique Belt away from the Kenya ALN for the Mozambique Belt away from the Kenya OUT for the Mozambique Belt away from the Fenya ERROR 4000 0.0664759 302 4543 4000 TRU its base (see Figure 1). ALN its base (see Figure 1). OUT its oase fsee iFigure i) 4100 TRU Three WM102 dual-frequency GPS receivers were used to track the GPS ALN Three WWM102 dual-frequency GPS receivers were used to track the GPS OUT Three MtO2 dual-freouency BPS receivers were used to track the 3PS 4200 TRU endless loop because in this strategy we use only the pick ALN endless loop because in this strategy we use only the pick OUT endless loop because in this strategy we use only the pick 4300 TRU [Tarabanis et al., 1991b] ALN [Tarabanis et al., 1991b] OUT TTarabanis ct ol., f91hl 4400 TRU 10%) relative to the much greater limitation on photo- ALN 10%%) relative to the much greater limitation on photo- OUT 1C9uo) relative to the much greater limitation on photo- 4500 TRU 6312 F. W. Bingham ALN 6312 F. W. Bingham OUT 6312 P. N. Bingham 4600 TRU Analysis and Machine Intelligence, 7(4):384-401, ALN Analysis and Machine Intelligence, 7(4):384-401, OUT Analgsis and Macbine Intelligence, TC4)i384-4D1 4700 TRU improves the fanout of the monitored signals. The phase- ALN improves the fanout of the monitored signals. The phase- OUT improves the fanout of the monitored signals. The phase- 4800 TRU "A Model-Based Computer Vision System ALN ""A Model-Based Computer Vision System OUT T7A Mocdel-3ased Computer Mision System 4900 TRU to send the prepared poll to the selected advisers or ALN to send the prepared poll to the selected advisers or OUT to send the prepared poll to the selected advisers or ERROR 5000 0.0490865 223 4543 5000 TRU instead of a series over time, to obtain ALN instead of a series over time, to obtain OUT instead of a series over time, to obtain 5100 TRU processors. ALN processors. OUT processors 5200 TRU loosen adhesion at the zone cell interface ALN loosen adhesion at the zone cell interface OUT loosen adhesion at the 2one cell interface 5300 TRU page). ALN page) OUT page) 5400 TRU problems. ALN problems. OUT problems 5500 TRU [Tarabanis et al., 1991b] ALN [Tarabanis et al., 1991b] OUT TTarabanis et ol., 1991hl 5600 TRU 3.2 Spatio-Geometric and Optical Compu- ALN 3.2 Spatio-Geometric and Optical Compu- OUT 3.2 Spatio-MGeometric and Optical Compu- 5700 TRU Such a coarsening mechanism can either be due to a ALN Such a coarsening mechanism can either be due to a OUT Such a coarsening mechanism can either be due to a 5800 TRU 11b. A fault scarp on the east is formed after ALN 11b. A fault scarp on the east is formed after OUT 11b. A fault scarp on the east is formed after