Here, you can see how you feed LIBSVM format data into CAFFE net.
%pylab inline
%cd ..
import os, shutil
dir_o = 'examples/tmp.libsvm_data'
! mkdir -p {dir_o}
Populating the interactive namespace from numpy and matplotlib /Users/takuya/Work/caffe
Create a temporary directory
We use the hand-written digits dataset provided by Scikit-Learn as an example. Image has a shape of (8, 8). The number of class is 10. The number of sample is 1797.
from sklearn.datasets import load_digits
digits = load_digits()
print digits.images.shape, digits.target.shape
(1797, 8, 8) (1797,)
Image is like this.
imshow(digits.images[0], cmap=gray(), interpolation='none')
<matplotlib.image.AxesImage at 0x107a2cb50>
Dataset is split into 2. We use first one for training and second one for testing.
def split2(arr):
div = arr.shape[0] / 2
return arr[:div], arr[div:]
data2 = split2(digits.data)
target2 = split2(digits.target)
print [d.shape[0] for d in data2]
[898, 899]
Here is just an example of training and testing of digits data using Logistic Regression provided by Scikit-Learn. The test accuracy is 91.7%.
from sklearn.linear_model import LogisticRegression
lr = LogisticRegression().fit(data2[0], target2[0])
from sklearn.metrics import accuracy_score
print 'Test Accuracy:', accuracy_score(target2[1], lr.predict(data2[1]))
Test Accuracy: 0.916573971079
Dataset will be saved as LIBSVM format in order to use from CAFFE. The format is as following.
<label> <feature index 1>:<feature value 1> ... <feature index N>:<feature value N>
def libsvm_format_generator(data, target):
from StringIO import StringIO
for datum, tar in zip(data, target):
sio = StringIO()
print >> sio, tar,
for i, v in enumerate(datum):
if v == 0: continue
print >> sio, '%d:%g' % (i, v),
yield sio.getvalue()
One digit image is written as follows.
print libsvm_format_generator(data2[0], target2[0]).next()
0 2:5 3:13 4:9 5:1 10:13 11:15 12:10 13:15 14:5 17:3 18:15 19:2 21:11 22:8 25:4 26:12 29:8 30:8 33:5 34:8 37:9 38:8 41:4 42:11 44:1 45:12 46:7 49:2 50:14 51:5 52:10 53:12 58:6 59:13 60:10
Write to files training and testing data respectively.
with open(os.path.join(dir_o, 'input_train.txt'), 'w') as f:
for l in libsvm_format_generator(data2[0], target2[0]):
print >> f, l
with open(os.path.join(dir_o, 'input_test.txt'), 'w') as f:
for l in libsvm_format_generator(data2[1], target2[1]):
print >> f, l
The following model works as Softmax Logistic Regression. To provide data from LIBSVM format data file, we use LIBSVM_DATA layer as below. You can specify input file by libsvm_data_param > source. You must set batch_size and channels (number of feature). You can set transofrm_param except crop_size and mirror.
%%writefile {dir_o}/linear_train_test.prototxt
name: "digits_libsvm"
layers {
name: "digits"
type: LIBSVM_DATA
top: "data"
top: "label"
libsvm_data_param {
source: "examples/tmp.libsvm_data/input_train.txt"
batch_size: 100
channels: 64
shuffle: true
}
transform_param {
scale: 0.00392156862745098
}
include: { phase: TRAIN }
}
layers {
name: "digits"
type: LIBSVM_DATA
top: "data"
top: "label"
libsvm_data_param {
source: "examples/tmp.libsvm_data/input_test.txt"
batch_size: 899
channels: 64
shuffle: false
}
transform_param {
scale: 0.00392156862745098
}
include: { phase: TEST }
}
layers {
name: "ip1"
type: INNER_PRODUCT
bottom: "data"
top: "ip1"
blobs_lr: 1
blobs_lr: 2
inner_product_param {
num_output: 64
weight_filler {
type: "gaussian"
std: 0.1
}
bias_filler {
type: "constant"
}
}
}
layers {
name: "accuracy"
type: ACCURACY
bottom: "ip1"
bottom: "label"
top: "accuracy"
include: { phase: TEST }
}
layers {
name: "loss"
type: SOFTMAX_LOSS
bottom: "ip1"
bottom: "label"
top: "loss"
}
Writing examples/tmp.libsvm_data/linear_train_test.prototxt
Solver is simple. No special setting.
%%writefile {dir_o}/linear_solver.prototxt
net: "examples/tmp.libsvm_data/linear_train_test.prototxt"
test_iter: 1
test_interval: 1000
base_lr: 1.0
momentum: 0.9
#weight_decay: 0.00001
lr_policy: "fixed"
display: 1000
max_iter: 4000
snapshot: 4000
snapshot_prefix: "examples/tmp.libsvm_data/linear"
solver_mode: CPU
random_seed: 0 # Just for reproducible purpose
Writing examples/tmp.libsvm_data/linear_solver.prototxt
!build/tools/caffe train --solver={dir_o}/linear_solver.prototxt 2>&1 | egrep -A1 "Iteration .*, Test"
I1212 21:28:49.545277 2074399488 solver.cpp:247] Iteration 0, Testing net (#0) I1212 21:28:49.561161 2074399488 solver.cpp:298] Test net output #0: accuracy = 0 -- I1212 21:28:50.505483 2074399488 solver.cpp:247] Iteration 1000, Testing net (#0) I1212 21:28:50.514246 2074399488 solver.cpp:298] Test net output #0: accuracy = 0.904338 -- I1212 21:28:51.252451 2074399488 solver.cpp:247] Iteration 2000, Testing net (#0) I1212 21:28:51.259482 2074399488 solver.cpp:298] Test net output #0: accuracy = 0.904338 -- I1212 21:28:51.994504 2074399488 solver.cpp:247] Iteration 3000, Testing net (#0) I1212 21:28:52.001040 2074399488 solver.cpp:298] Test net output #0: accuracy = 0.917686 -- I1212 21:28:52.689991 2074399488 solver.cpp:247] Iteration 4000, Testing net (#0) I1212 21:28:52.697223 2074399488 solver.cpp:298] Test net output #0: accuracy = 0.925473
The test accuracy was 92.5 % which was similar result to ScikitLearn one.
Just an example
%%writefile {dir_o}/mlp_train_test.prototxt
name: "digits_libsvm"
layers {
name: "digits"
type: LIBSVM_DATA
top: "data"
top: "label"
libsvm_data_param {
source: "examples/tmp.libsvm_data/input_train.txt"
batch_size: 100
channels: 64
shuffle: true
}
transform_param {
scale: 0.00392156862745098
}
include: { phase: TRAIN }
}
layers {
name: "digits"
type: LIBSVM_DATA
top: "data"
top: "label"
libsvm_data_param {
source: "examples/tmp.libsvm_data/input_test.txt"
batch_size: 899
channels: 64
shuffle: false
}
transform_param {
scale: 0.00392156862745098
}
include: { phase: TEST }
}
layers {
name: "ip1"
type: INNER_PRODUCT
bottom: "data"
top: "ip1"
blobs_lr: 1
blobs_lr: 2
inner_product_param {
num_output: 64
weight_filler {
type: "gaussian"
std: 0.1
}
bias_filler {
type: "constant"
}
}
}
layers {
name: "relu1"
type: RELU
bottom: "ip1"
top: "ip1"
}
layers {
name: "ip2"
type: INNER_PRODUCT
bottom: "ip1"
top: "ip2"
blobs_lr: 1
blobs_lr: 2
inner_product_param {
num_output: 10
weight_filler {
type: "gaussian"
std: 0.1
}
bias_filler {
type: "constant"
}
}
}
layers {
name: "accuracy"
type: ACCURACY
bottom: "ip2"
bottom: "label"
top: "accuracy"
include: { phase: TEST }
}
layers {
name: "loss"
type: SOFTMAX_LOSS
bottom: "ip2"
bottom: "label"
top: "loss"
}
Writing examples/tmp.libsvm_data/mlp_train_test.prototxt
%%writefile {dir_o}/mlp_solver.prototxt
net: "examples/tmp.libsvm_data/mlp_train_test.prototxt"
test_iter: 1
test_interval: 1000
base_lr: 0.1
momentum: 0.9
weight_decay: 0.00001
lr_policy: "fixed"
display: 1000
max_iter: 4000
snapshot: 4000
snapshot_prefix: "examples/tmp.libsvm_data/mlp"
solver_mode: CPU
random_seed: 0 # Just for reproducible purpose
Writing examples/tmp.libsvm_data/mlp_solver.prototxt
!build/tools/caffe train --solver={dir_o}/mlp_solver.prototxt 2>&1 | egrep -A1 "Iteration .*, Test"
I1212 21:28:53.153337 2074399488 solver.cpp:247] Iteration 0, Testing net (#0) I1212 21:28:53.157271 2074399488 solver.cpp:298] Test net output #0: accuracy = 0.18465 -- I1212 21:28:53.533761 2074399488 solver.cpp:247] Iteration 1000, Testing net (#0) I1212 21:28:53.536794 2074399488 solver.cpp:298] Test net output #0: accuracy = 0.897664 -- I1212 21:28:53.946993 2074399488 solver.cpp:247] Iteration 2000, Testing net (#0) I1212 21:28:53.951678 2074399488 solver.cpp:298] Test net output #0: accuracy = 0.92436 -- I1212 21:28:54.401706 2074399488 solver.cpp:247] Iteration 3000, Testing net (#0) I1212 21:28:54.404630 2074399488 solver.cpp:298] Test net output #0: accuracy = 0.92881 -- I1212 21:28:54.811215 2074399488 solver.cpp:247] Iteration 4000, Testing net (#0) I1212 21:28:54.814102 2074399488 solver.cpp:298] Test net output #0: accuracy = 0.941046
Remove temporary directory
!rm -rf {dir_o}