8 PLEAC Julia: File Contents¶

PLEAC = Language Examples Alike Cookbook PLEAC examples are drawn from the "Perl Cookbook" by Tom Christiansen & Nathan Torkington, published by O'Reilly. They provide a nice range of examples oriented toward data munging, the type of work I tend to want to do first when learning a new language.

The Julia examples below are principally translations from the Python version

Caution¶

I'm learning as I go, so the code below probably doesn't represent best practice. Your suggestions are welcome! Please file an issue or make a pull request to the github repo.

The examples are not complete. Missing items are generally noted in comments.

8.0 Introduction¶

In [1]:

#make a file
contents="""Here are
a few
lines of
text content."""

fo=open("julia_file.txt", "w")
write(fo, contents)
close(fo)

In [2]:

fi=open("julia_file.txt", "r")
for line in readlines(fi)
  line = rstrip(line)
  size = length(line)
  println(size)
end
close(fi)

In [3]:

fi=open("julia_file.txt", "r")
for line in readlines(fi)
  println(length(rstrip(line)))
end
close(fi)

In [4]:

al=readlines(open("julia_file.txt", "r"))

Out[4]:

4-element Array{Any,1}:
 "Here are\n"   
 "a few\n"      
 "lines of\n"   
 "text content."

In [5]:

t=readall(open("julia_file.txt", "r"))

Out[5]:

"Here are\na few\nlines of\ntext content."

In [6]:

Out[6]:

"Here are\na few\nlines of\ntext content."

In [7]:

w=readdlm(open("julia_file.txt", "r"), ' ')[1] #first word

Out[7]:

"Here"

In [8]:

fi=open("julia_file.txt2", "w")
writedlm(fi, ["One","Two","Three"], ' ')
close(fi)

In [9]:

help(seek)

Loading help data...
Base.seek(s, pos)

   Seek a stream to the given position.

In [10]:

fi=open("julia_file.txt", "r")
ba=readbytes(fi, 4096)
ba'

Out[10]:

1x37 Array{Uint8,2}:
 0x48  0x65  0x72  0x65  0x20  0x61  …  0x6e  0x74  0x65  0x6e  0x74  0x2e

In [11]:

seek(fi,12)  #seek to byte 12
println("I'm $(position(fi)) bytes from the start of the file")

I'm 12 bytes from the start of the file

In [12]:

seekstart(fi) #seek to start
seekend(fi) #seek to end

Out[12]:

IOStream(<file julia_file.txt>)

In [13]:

seekend(fi)
skip(fi, -2)  #relative seek
position(fi)

Out[13]:

In [14]:

position(fi)

Out[14]:

In [15]:

# NOT IMPLEMENTED:
# pos = os.lseek(myfile.fileno(),0,1)  #?? what does this do?

8.1 Reading Lines with Continuation Characters¶

In [16]:

#not implemented

8.2 Counting Lines (or Paragraphs or Records) in a File¶

In [17]:

open("julia_file.txt", "r") do fi
   lines= readlines(fi)
end
length(lines)

WARNING:

lines not defined
at In[17]:4

In [18]:

t=readall(open("julia_file.txt", "r"))

backtraces on your platform are often misleading or partially incorrect

Out[18]:

"Here are\na few\nlines of\ntext content."

In [19]:

para_count = length(split(t, "in"))

Out[19]:

8.3 Processing Every Word in a File¶

In [20]:

#simple word count
words=split(t)

Out[20]:

8-element Array{String,1}:
 "Here"    
 "are"     
 "a"       
 "few"     
 "lines"   
 "of"      
 "text"    
 "content."

In [21]:

word_dict=Dict()
for w in words
  word_dict[w] = get(word_dict, w, 0)+1
end
word_dict

Out[21]:

{"text"=>1,"content."=>1,"are"=>1,"few"=>1,"Here"=>1,"lines"=>1,"of"=>1,"a"=>1}

Cleaner results using TextAnalysis¶

In [22]:

using TextAnalysis

In [23]:

tokens(Document(t))

Out[23]:

8-element Array{UTF8String,1}:
 "Here"    
 "are"     
 "a"       
 "few"     
 "lines"   
 "of"      
 "text"    
 "content."

In [24]:

words=split(remove_punctuation!(Document(t)))  #we could remove numbers too.

Out[24]:

8-element Array{String,1}:
 "Here"   
 "are"    
 "a"      
 "few"    
 "lines"  
 "of"     
 "text"   
 "content"

In [25]:

word_dict=Dict()
for w in words
  word_dict[w] = get(word_dict, w, 0)+1
end
word_dict

Out[25]:

{"text"=>1,"are"=>1,"few"=>1,"Here"=>1,"lines"=>1,"content"=>1,"of"=>1,"a"=>1}

In [26]:

#line frequency count
lines=readlines(open("julia_file.txt", "r"))
line_dict=Dict()
for w in lines
  line_dict[w] = get(line_dict, w, 0)+1
end
line_dict

Out[26]:

{"lines of\n"=>1,"Here are\n"=>1,"a few\n"=>1,"text content."=>1}

8.4 Reading a File Backwards by Line or Paragraph¶

In [27]:

lines= reverse( readlines(open("julia_file.txt", "r")) )

Out[27]:

4-element Array{Any,1}:
 "text content."
 "lines of\n"   
 "a few\n"      
 "Here are\n"

8.5 Trailing a Growing File¶

In [28]:

fi=open("julia_file.txt", "r")
seekend(fi)
endpos = position(fi)

Out[28]:

In [29]:

i=1
while i<3 #or while true
  sleep(1)
  seekend(fi)
  if position(fi)>endpos
    endpos=position(fi)
    println("time: $i. new size: $endpos")
  end
  i=i+1
end

In [30]:

close(fi)

8.6 Picking a Random Line from a File¶

In [31]:

lines= reverse( readlines(open("julia_file.txt", "r")) )
length(lines)

Out[31]:

In [32]:

pick = rand(1:length(lines))
lines[pick]

Out[32]:

"Here are\n"

8.7 Randomizing All Lines¶

In [33]:

using Distributions

In [34]:

shuffle(lines)

Out[34]:

4-element Array{Any,1}:
 "a few\n"      
 "text content."
 "Here are\n"   
 "lines of\n"

8.8 Reading a Particular Line in a File¶

In [35]:

DESIRED_LINE=3
lines[DESIRED_LINE]

Out[35]:

"a few\n"

8.9 Processing Variable-Length Text Fields¶

In [36]:

pattern=" "
record = lines[1]
split(record)

Out[36]:

2-element Array{String,1}:
 "text"    
 "content."

8.10 Removing the Last Line of a File¶

In [37]:

#create test file
#make a file
contents="""Here are
a few
lines of
text content."""

fo=open("julia_file.txt", "w")
write(fo, contents)
close(fo)

In [38]:

fi=open("julia_file.txt", "r+")
pos=0
prev_pos=0
for ln in eachline(fi)
  prev_pos = pos
  pos = position(fi)
end
close(fi)
println(prev_pos)

Warning: could not import Base.foldl into NumericExtensions

Warning: could not import Base.foldr into NumericExtensions

In [39]:

cp("julia_file.txt", "julia_file2.txt")
open("julia_file2.txt", "w") do fi
    truncate(fi, prev_pos)
end

trunc8 = readall(open("julia_file.txt", "r"))
open("julia_file2.txt", "w") do fo
    write(fo, trunc8[1:prev_pos])
end

result = readall(open("julia_file2.txt", "r"))
result

Out[39]:

"Here are\na few\nlines of\n"

8.11 Processing Binary Files¶

In [40]:

open("julia_file.txt", "r") do fi
    ba=readbytes(fi)
end
ba'

Out[40]:

1x37 Array{Uint8,2}:
 0x48  0x65  0x72  0x65  0x20  0x61  …  0x6e  0x74  0x65  0x6e  0x74  0x2e

8.12 Using Random-Access I/O¶

In [41]:

function readstring(stream, numchars)
  bytes=readbytes(stream, numchars)
  return join([char(v) for v in bytes])
end

Out[41]:

readstring (generic function with 1 method)

In [42]:

#make a file : 2 fields of 4 chars (+ \n) = 9 characters per line
contents="""   a   b
blah  at
line  of
text1234"""

open("julia_fixed.dat", "w") do fo
    write(fo, contents)
end

Out[42]:

In [43]:

recsize=8+1  # 8 characters of data + line ending
recno=2
address = recsize * recno 
open("julia_fixed.dat", "r") do fi
    seek(fi,address)
    println(readstring(fi, recsize))
end

line  of

8.13 Updating a Random-Access File¶

In [52]:

open("julia_fixed.dat", "r+") do fi
    seek(fi,address)
    recstr = readstring(fi, recsize)
    newrec = uppercase(recstr)
    seek(fi,address)
    write(fi,newrec)
end

Out[52]:

In [54]:

open("julia_fixed.dat", "r") do fi
    dt=readall(fi)
    print(dt)
end

   a   b
blah  at
LINE  OF
text1234

8.14 Reading a String from a Binary File¶

In [46]:

#NOT IMPLEMENTED

8.15 Reading Fixed-Length Records¶

In [85]:

Pkg.add("IniFile")

INFO: No packages to install, update or remove.
INFO: REQUIRE updated.

In [71]:

# define a packable binary record
using StrPack
@struct  type TstStr
    int1::Int64
    float1::Float64
end
RECORDSIZE=sizeof(TstStr)

Out[71]:

In [81]:

#write out sample data
rec=TstStr(4455, 556.32)
open("bintest.dat","w") do fo
    pack(fo, rec)
    pack(fo, TstStr(3455, 556.32))
    pack(fo, TstStr(6453, 656.32))    
end

WARNING:

max(x) is deprecated, use maximum(x) instead.
 in depwarn at deprecated.jl:29
 in string at ascii.jl:35
WARNING: max(x) is deprecated, use maximum(x) instead.
 in depwarn at deprecated.jl:29
 in string at ascii.jl:35
WARNING: max(x) is deprecated, use maximum(x) instead.
 in depwarn at deprecated.jl:29
 in string at ascii.jl:35

Out[81]:

In [84]:

open("bintest.dat","r") do fi
    seek(fi, RECORDSIZE)   # "rewind" to the beginning of the buffer
    s2 = unpack(fi, TstStr)
    println(s2)
end

WARNING:

TstStr(3455,556.32)

max(x) is deprecated, use maximum(x) instead.
 in depwarn at deprecated.jl:29
 in string at ascii.jl:35

8.16 Reading Configuration Files¶

In [87]:

using IniFile
ini = Inifile()
read(ini, "test.ini")
get(ini, "section 1", "strname1") #== "section1name1"

Out[87]:

"section1name1"

8.17 Testing a File for Trustworthiness¶

In [49]:

#Will not implement

Programs¶

In [50]:

#Will not implement