1. PLEAC Julia: Strings¶

PLEAC = Language Examples Alike Cookbook PLEAC examples are drawn from the "Perl Cookbook" by Tom Christiansen & Nathan Torkington, published by O'Reilly. They provide a nice range of examples oriented toward data munging, the type of work I tend to want to do first when learning a new language.

The Julia examples below are principally translations from the Python version

Caution¶

I'm learning as I go, so the code below probably doesn't represent best practice. Your suggestions are welcome! Please file an issue or make a pull request to the github repo.

Why isn't this in the main PLEAC repo?¶

IJulia_PLEAC uses IJulia notebook, whose format appears to be incompatible with the PLEAC file format.

The examples are not complete. Missing items are generally noted in comments.

PLEAC 1.0: Introduction¶

Let's try out some string literals.

In [12]:

mystr = "\n"      # a string containing 1 character, which is a newline

Out[12]:

"\n"

In [13]:

mystr = "\\\n"  # two characters, \ and n

Out[13]:

"\\\n"

In [14]:

length(mystr)==2

Out[14]:

true

In [15]:

mystr = "Jon 'Maddog' Orwant"  # literal single quote inside double quotes

Out[15]:

"Jon 'Maddog' Orwant"

In [16]:

mystr = """Jon "Maddog" Orwant"""  # literal double quote inside triple quotes

Out[16]:

"Jon \"Maddog\" Orwant"

In [17]:

mystr = "Jon \"Maddog\" Orwant"  # escaped double quote

Out[17]:

"Jon \"Maddog\" Orwant"

In [18]:

mystr = """
This is a multiline string literal
enclosed in triple double quotes.
"""

Out[18]:

"This is a multiline string literal\nenclosed in triple double quotes.\n"

In [155]:

#julia does not currently have an equivalent to python docstringss, but triple-quoted comments are allowed
function fn(x)
   """we can put a comment here"""
   return 2x
end

fn(3)

Out[155]:

PLEAC 1.1: Accessing Substrings¶

In [19]:

offset=2; count=3;
mystr[offset:offset+count]

Out[19]:

"his "

In [20]:

mystr[offset:]  # offset to end

Out[20]:

"his is a multiline string literal\nenclosed in triple double quotes.\n"

Get a 5-char string, skip 3, then grab 2 8-char strings, then the rest¶

In [21]:

data = b"abcdefghijklmnopqrstuvwxyz";  # byte array 0x61-0x7a
leading, s1, s2, trailing = data[1:5], data[9:16], data[17:24], data[25:]

Out[21]:

([0x61,0x62,0x63,0x64,0x65],[0x69,0x6a,0x6b,0x6c,0x6d,0x6e,0x6f,0x70],[0x71,0x72,0x73,0x74,0x75,0x76,0x77,0x78],[0x79,0x7a])

Split at five-char boundaries¶

In [22]:

typeof(data)

Out[22]:

Array{Uint8,1}

In [23]:

a=Array{Uint8,1}[]

Out[23]:

0-element Array{Array{Uint8,1},1}

In [24]:

push!(a,data[1:5])

Out[24]:

1-element Array{Array{Uint8,1},1}:
 [0x61,0x62,0x63,0x64,0x65]

In [25]:

i=1
count=5
fivers=Array{Uint8,1}[]  # 1 dimensional array of bytearrays
while i<=length(data)
   chunk = data[ i : min(i+4,length(data)) ]
   push!(fivers, chunk )
   i=i+count
end
fivers

Out[25]:

6-element Array{Array{Uint8,1},1}:
 [0x61,0x62,0x63,0x64,0x65]
 [0x66,0x67,0x68,0x69,0x6a]
 [0x6b,0x6c,0x6d,0x6e,0x6f]
 [0x70,0x71,0x72,0x73,0x74]
 [0x75,0x76,0x77,0x78,0x79]
 [0x7a]

index to sub-string¶

In [26]:

mystr = "This is what you have"
# julia indexes are 1-based
first = mystr[1]                            # "T"

Out[26]:

'T'

In [27]:

start = mystr[6:7]                          # "is"

Out[27]:

"is"

In [28]:

rest = mystr[14:]                           # "you have"

Out[28]:

"you have"

In [29]:

last = mystr[end:]                            # "e" a string

Out[29]:

"e"

In [30]:

mystr[end] #a char

Out[30]:

'e'

In [31]:

mystr[end-1] #a char

Out[31]:

'v'

In [32]:

end4 = mystr[end-3:]                            # "have"

Out[32]:

"have"

In [33]:

piece = mystr[end-7:end-4]                        # "you"

Out[33]:

"you "

In [34]:

mystr[1:10]

Out[34]:

"This is wh"

In [35]:

mystr[10+1:end]

Out[35]:

"at you have"

replace substr¶

In [36]:

replace(mystr, " is ", " wasn't ")

Out[36]:

"This wasn't what you have"

test for substr¶

In [37]:

txt = "at"
contains(mystr,txt)

Out[37]:

true

In [38]:

beginswith(mystr, txt)

Out[38]:

false

In [39]:

endswith(mystr,txt)

Out[39]:

false

PLEAC 1.2: Establishing a Default Value¶

Background - checking for state of a variable.¶

In [40]:

isdefined(:astr) 

Out[40]:

false

In [41]:

astr=None

Out[41]:

None

In [42]:

astr == None  

Out[42]:

true

In [43]:

isdefined(:astr)  # defined but None

Out[43]:

true

In [44]:

:(astr)

Out[44]:

:astr

Haven't found an existing function to do this:¶

In [45]:

function isassigned(myvar::Symbol)
    """return true if the Symbol is defined and not None"""
    DEBUG=false
    if DEBUG println( :(myvar) ) end
    if DEBUG println( isdefined(myvar) ) end
    if isdefined(myvar)        
        varval = eval(myvar);
        if DEBUG println( "varval $varval" ) end
        if varval==None            
            if DEBUG println( "$myvar==None ") end
            return false
        elseif varval==""            
            if DEBUG println( """$myvar=="" """) end
            return false
        else
            if DEBUG println("$myvar has an assigned value") end
            return true
        end
    else
        if DEBUG println("not defined") end
        return false
    end
end

Out[45]:

isassigned (generic function with 1 method)

In [46]:

isassigned( :nosuchvar )

Out[46]:

false

In [47]:

isassigned( :astr )

Out[47]:

false

In [48]:

ex=999
isassigned(:ex)

Out[48]:

true

Setting a default value¶

In [49]:

mydefault = "blah"

Out[49]:

"blah"

In [50]:

ystr =  isdefined(:(astr)) & (astr!=None) ? astr : mydefault 
ystr

Out[50]:

"blah"

In [51]:

ystr = isassigned(:astr) ? astr : mydefault

Out[51]:

"blah"

if myvar is returned from a function and may be empty/None, then use:¶

In [52]:

function f(x::Int)
   """a function that can return None""" 
   y = (x==4) ? x : None 
   return y
end

Out[52]:

f (generic function with 1 method)

In [53]:

myvar = f(3)
myvar== (myvar==None) ? "blah" : myvar

Out[53]:

None

if you want a default value that can be overridden by the person calling your code, you can often wrap it in a function with a named parameter:¶

In [54]:

function fdef(prefix="pre")
   """a function that can return None""" 
   prefix * " duh"
end

Out[54]:

fdef (generic function with 2 methods)

In [55]:

fdef("asd")

Out[55]:

"asd duh"

In [56]:

fdef()

Out[56]:

"pre duh"

In [57]:

# use b if b is defined, else c
a = isdefined(:b) ? b : 21

Out[57]:

In [58]:

ENV["USERNAME"]

Out[58]:

"KEITHC"

In [59]:

haskey(ENV,"USERNAME")

Out[59]:

true

In [60]:

function get_username()
    if haskey(ENV,"USERNAME") 
        usr=ENV["USERNAME"]
    elseif haskey(ENV,"LOGNAME") 
        usr=ENV["LOGNAME"]
    elseif haskey(ENV,"USER") 
        usr=ENV["USER"]    
    else 
        usr = None
    end
    return usr
end

Out[60]:

get_username (generic function with 1 method)

In [61]:

get_username()

Out[61]:

"KEITHC"

PLEAC 1.3: Exchanging values without Using Temporary Variables¶

In [61]:

# v1, v2 = v2, v1
x, y = (4, 3);

In [62]:

Out[62]:

In [63]:

Out[63]:

In [64]:

one, two, three = split("January March August")

Out[64]:

3-element Array{String,1}:
 "January"
 "March"  
 "August"

In [65]:

[one, two, three]'

Out[65]:

1x3 Array{ASCIIString,2}:
 "January"  "March"  "August"

In [66]:

one, two, three = two, three, one

Out[66]:

("March","August","January")

In [67]:

one

Out[67]:

"March"

PLEAC 1.4: Converting between ASCII Characters and Values¶

In [68]:

char = 'a'

Out[68]:

'a'

In [69]:

ascii_value = int(char)

Out[69]:

In [70]:

Base.char(120)  # char(120) should work but I'm getting a namespace collision

Out[70]:

'x'

In [71]:

println("number $(int(char)) is character '$char'")

number 97 is character 'a'

In [72]:

ascii_character_numbers = [int(c) for c in "sample"]

Out[72]:

6-element Array{Int64,1}:
 115
  97
 109
 112
 108
 101

In [73]:

join([c for c in "sample"],"")

Out[73]:

"sample"

In [74]:

join([Base.char(i) for i in ascii_character_numbers],"")

Out[74]:

"sample"

In [75]:

hal="HAL"
join([Base.char(int(i)+1) for i in hal],"")

Out[75]:

"IBM"

PLEAC 1.5: Processing a String one Character at a Time¶

In [76]:

myarray = [c for c in "whatever"]'

Out[76]:

1x8 Array{Char,2}:
 'w'  'h'  'a'  't'  'e'  'v'  'e'  'r'

In [77]:

for mychar in "whatever"
   println(mychar)
end

w
h
a
t
e
v
e
r

In [78]:

cset = Set([c for c in "Whatever"]...)

Out[78]:

Set{Char}('r','a','h','W','v','t','e')

In [79]:

split("whatever","")'

Out[79]:

1x8 Array{String,2}:
 "w"  "h"  "a"  "t"  "e"  "v"  "e"  "r"

In [80]:

sset =Set( split("whatever","")... )

Out[80]:

Set{ASCIIString}("a","e","r","v","w","t","h")

In [81]:

csort = sort([c for c in cset])'

Out[81]:

1x7 Array{Any,2}:
 'W'  'a'  'e'  'h'  'r'  't'  'v'

In [82]:

println("unique chars are:   ", join(csort,","))

unique chars are:   W,a,e,h,r,t,v

In [82]:

mystr="whatever"
ascvals= [int(c) for c in mystr];

In [83]:

println("""Total is $(sum(ascvals)) for "$mystr".""")

Total is 870 for "whatever".

In [84]:

function checksum(mystr) 
    vals = [ int(c) for c in mystr ]
    chksum = sum(vals) % (2^16) - 1
    return chksum
end

Out[84]:

checksum (generic function with 1 method)

In [85]:

checksum("whatever")

Out[85]:

In [86]:

fl = open("c:/temp/blah.txt","w")
write(fl, "what up, gee?")
close(fl)

In [87]:

flstr = readall("c:/temp/blah.txt")

Out[87]:

"what up, gee?"

In [88]:

checksum(flstr)

Out[88]:

PLEAC 1.6: Reversing a String by Word or Character¶

In [89]:

reverse([1,2,3])

Out[89]:

3-element Array{Int64,1}:
 3
 2
 1

In [90]:

reverse("Stuff")

Out[90]:

"ffutS"

In [91]:

mystr="a bunch of words"
myarray = split(mystr)

Out[91]:

4-element Array{String,1}:
 "a"    
 "bunch"
 "of"   
 "words"

In [92]:

join(reverse(myarray), " ")

Out[92]:

"words of bunch a"

In [93]:

word = "reviver"
is_palindrome = (word == reverse(word))

Out[93]:

true

skipped long palindromes

PLEAC 1.7: Expanding and Compressing Tabs¶

In [94]:

replace("adfa\t    ",'\t',"   ")

Out[94]:

"adfa       "

In [95]:

replace("        blah", "    ",'\t')

Out[95]:

"\t\tblah"

PLEAC 1.8: Expanding Variables in User Input¶

In [96]:

#-----------------------------
rows=24; cols=80
text = "I am $(rows) high and $(cols) long"
print( text)

I am 24 high and 80 long

In [97]:

re = r"\d+"

Out[97]:

r"\d+"

In [98]:

ismatch(re,"Are you 43?")

Out[98]:

true

In [99]:

ismatch(re,"Are you XXIII?")

Out[99]:

false

In [100]:

m=match(re,"Are you 43?")

Out[100]:

RegexMatch("43")

In [101]:

dump(m)

RegexMatch 
  match: SubString{UTF8String} "43"
  captures: Array(Union(Nothing,SubString{UTF8String}),(0,)) []
  offset: Int64 9
  offsets: Array(Int64,(0,)) []

In [102]:

m.offset

Out[102]:

In [103]:

m.match

Out[103]:

"43"

In [104]:

replace("Are you 23 or 43?",re,"xx")

Out[104]:

"Are you xx or xx?"

In [105]:

m.captures

Out[105]:

0-element Array{Union(Nothing,SubString{UTF8String}),1}

In [106]:

rec=r"(\d+)"

Out[106]:

r"(\d+)"

In [107]:

m=match(rec,"Are you 43?")
dump(m)

RegexMatch 
  match: SubString{UTF8String} "43"
  captures: Array(Union(Nothing,SubString{UTF8String}),(1,)) ["43"]
  offset: Int64 9
  offsets: Array(Int64,(1,)) [9]

In [108]:

parseint( m.captures[1])

Out[108]:

In [109]:

mystr = "Are you 43?"
m = match(rec,"Are you 43?")
replace(mystr, m.captures[1], string(2*parseint(m.captures[1])) )

Out[109]:

"Are you 86?"

Safe Substitution¶

In [110]:

function default_string(string_symbol::Symbol, default="[Not Defined]")
    def_str=  isdefined(string_symbol) ? eval(string_symbol) : "[Not Defined]"
    return def_str
end

Out[110]:

default_string (generic function with 2 methods)

In [111]:

"""who is $(isdefined(:z) ? z : "[Not Defined]")"""

Out[111]:

"who is [Not Defined]"

In [112]:

"""who is $(default_string(:z))"""

Out[112]:

"who is [Not Defined]"

PLEAC 1.9: Controlling Case¶

In [113]:

uppercase("bo peep")

Out[113]:

"BO PEEP"

In [114]:

lowercase("Bo Peep")

Out[114]:

"bo peep"

In [115]:

mystr="bo Peep"

Out[115]:

"bo Peep"

In [116]:

function capitalcase(mystr)
    """capitalize the first letter of each word in a string"""
    return join([ucfirst(s) for s in split(mystr)], " ")
end

Out[116]:

capitalcase (generic function with 1 method)

In [117]:

capitalcase("bo peep went to fetch")

Out[117]:

"Bo Peep Went To Fetch"

In [118]:

ucfirst("bo peep")

Out[118]:

"Bo peep"

In [119]:

#skipped random case letters

PLEAC 1.10: Interpolating Functions and Expressions Within Strings¶

In [120]:

n=45
"I have $(n + 1) guanacos." 

Out[120]:

"I have 46 guanacos."

Use templates from Package Mustache.jl to substitute from a dictionary.

In [121]:

using Mustache

In [122]:

mytemplate = mt"
To:   {{address}}
From: Your Bank
CC:   {{cc_number}}
Date: {{date}}

Dear {{name}},

Today you bounced check number {{checknum}} to us.
Your account is now closed.

Sincerely,
the management
"

Out[122]:

MustacheTokens({{"text","\nTo:   ",0,7},{"name","address",7,20},{"text","\nFrom: Your Bank\nCC:   ",20,43},{"name","cc_number",43,58},{"text","\nDate: ",58,65},{"name","date",65,75},{"text","\n\nDear ",75,82},{"name","name",82,92},{"text",",\n\nToday you bounced check number ",92,126},{"name","checknum",126,140},{"text"," to us.\nYour account is now closed.\n\nSincerely,\nthe management\n",140,203}})

In [123]:

person = {"address"=>"Joe@somewhere.com",
          "name"=>"Joe",
          "date"=>"2012-04-22",
          "cc_number"=>1234567890,
          "checknum"=> 512          
          }

Out[123]:

{"date"=>"2012-04-22","checknum"=>512,"cc_number"=>1234567890,"address"=>"Joe@somewhere.com","name"=>"Joe"}

In [124]:

print(render(mytemplate, person))

To:   Joe@somewhere.com
From: Your Bank
CC:   1234567890
Date: 2012-04-22

Dear Joe,

Today you bounced check number 512 to us.
Your account is now closed.

Sincerely,
the management

PLEAC 1.11: Indenting Here Documents¶

indenting here documents
not clear we need to do anything --there are no leading blanks by default

In [125]:

var = """
      your text
      goes here
      """

Out[125]:

"your text\ngoes here\n"

In [126]:

print(var)

your text
goes here

split into lines, use every line except first and last, left strip and rejoin.¶

In [127]:

poem = """
       Here's your poem:
       Now far ahead the Road has gone,
          And I must follow, if I can,
       Pursuing it with eager feet,
          Until it joins some larger way
       Where many paths and errand meet.
          And whither then? I cannot say.
               --Bilbo in /usr/src/perl/pp_ctl.c  
       """

Out[127]:

"Here's your poem:\nNow far ahead the Road has gone,\n   And I must follow, if I can,\nPursuing it with eager feet,\n   Until it joins some larger way\nWhere many paths and errand meet.\n   And whither then? I cannot say.\n        --Bilbo in /usr/src/perl/pp_ctl.c  \n"

In [128]:

print( poem)

Here's your poem:
Now far ahead the Road has gone,
   And I must follow, if I can,
Pursuing it with eager feet,
   Until it joins some larger way
Where many paths and errand meet.
   And whither then? I cannot say.
        --Bilbo in /usr/src/perl/pp_ctl.c

In [129]:

print( join( [lstrip(ln) for ln in split(poem,'\n')[2:end-2] ], "\n"))

Now far ahead the Road has gone,
And I must follow, if I can,
Pursuing it with eager feet,
Until it joins some larger way
Where many paths and errand meet.
And whither then? I cannot say.

PLEAC 1.12: Reformatting Paragraphs¶

In [131]:

require("textwrap")
using TextWrap

In [132]:

txt = """\
Folding and splicing is the work of an editor,
not a mere collection of silicon
and
mobile electrons!
"""

Out[132]:

"Folding and splicing is the work of an editor,\nnot a mere collection of silicon\nand\nmobile electrons!\n"

In [133]:

print(TextWrap.wrap(txt, width=20, initial_indent=4, subsequent_indent=2))

    Folding and
  splicing is the
  work of an editor,
  not a mere
  collection of
  silicon and mobile
  electrons!

In [133]:

"""Expected result:

01234567890123456789
    Folding and
  splicing is the
  work of an editor,
  not a mere
  collection of
  silicon and mobile
  electrons!
""";

In [134]:

# merge multiple lines into one, then wrap one long line -- SKIP

PLEAC 1.13: Escaping Characters¶

Will not solve.

PLEAC 1.14: Trimming Blanks from the Ends of a String¶

In [135]:

mystr = "  blah  "

Out[135]:

"  blah  "

In [136]:

lstrip(mystr)

Out[136]:

"blah  "

In [137]:

rstrip(mystr)

Out[137]:

"  blah"

In [138]:

strip(mystr)

Out[138]:

"blah"

PLEAC 1.15: Parsing Comma-Separated Data¶

In [139]:

ezcsv = """a,b,c
           1,2,3
           4,5,6"""

Out[139]:

"a,b,c\n1,2,3\n4,5,6"

In [140]:

line = """XYZZY,"","O'Reilly, Inc","Wall, Larry","a \\"glug\\" bit,",5,"Error, Core Dumped,","""

Out[140]:

"XYZZY,\"\",\"O'Reilly, Inc\",\"Wall, Larry\",\"a \\\"glug\\\" bit,\",5,\"Error, Core Dumped,\","

In [141]:

sio = IOBuffer(ezcsv)

Out[141]:

IOBuffer([0x61,0x2c,0x62,0x2c,0x63,0x0a,0x31,0x2c,0x32,0x2c,0x33,0x0a,0x34,0x2c,0x35,0x2c,0x36],true,false,true,false,17,9223372036854775807,1)

In [142]:

data_array =readdlm(sio, ',')

Out[142]:

3x3 Array{Any,2}:
  "a"   "b"   "c"
 1.0   2.0   3.0 
 4.0   5.0   6.0

In [143]:

close(sio)

In [144]:

function stringdlm(source::String, delim::Char; has_header=false, use_mmap=false, ignore_invalid_chars=false)
    """parse a string of delimited text"""
    sio = IOBuffer(source)
    data_array =Base.readdlm(sio, ',')
    close(sio)
    return data_array
end 

Out[144]:

stringdlm (generic function with 1 method)

In [145]:

stringdlm(ezcsv,',')

Out[145]:

3x3 Array{Any,2}:
  "a"   "b"   "c"
 1.0   2.0   3.0 
 4.0   5.0   6.0

readdlm() does not appear to know about quoted data, so it does not handle embedded delimiters well¶

For example "Wall, Larry" should not have been split below.

In [146]:

#bug in readdlm()?
for ln in stringdlm(line,',')'
  println(ln)
end

XYZZY
""
"O'Reilly
 Inc"
"Wall
 Larry"
"a \"glug\" bit
"
5.0
"Error
 Core Dumped
"

In [147]:

stringdlm("""'Bush,Reggie','Morris,Alfred'""", ',')

Out[147]:

1x3 Array{Any,2}:
 "'Bush"  "Reggie'"  "'Morris"

here are a couple of simple test files to demonstrate treatment of quoted data¶

test0.csv:

"Name","Age","Weight"
"Bob",26,160
"Sue",24,110

test1.csv -- the name field includes embedded commas:

"Name","Age","Weight"
"Jones,Bob",26,160
"Collins,Sue",24,110

readdlm() does not seem to know about quotes around strings.

In [148]:

readdlm("test0.csv",',')

WARNING:

new not defined
at In[148]:1
 in mmap_array at mmap.jl:133

as a consequence, readdlm() has trouble with test1.csv:

In [149]:

readdlm("test1.csv",',')

backtraces on your platform are often misleading or partially incorrect

new not defined
at In[149]:1
 in mmap_array at mmap.jl:133

Package DataFrames contains as readtable() function that is smarter¶

However, it does not appear to have an option to using an IOBuffer at the moment.

In [150]:

using DataFrames

In [151]:

readtable("test0.csv")

Out[151]:

2x3 DataFrame:
         Name Age Weight
[1,]    "Bob"  26    160
[2,]    "Sue"  24    110

In [152]:

readtable("test1.csv")

Out[152]:

2x3 DataFrame:
                 Name Age Weight
[1,]      "Jones,Bob"  26    160
[2,]    "Collins,Sue"  24    110

PLEAC 1.16-1.18 Soundex and Programs fixstyle, psgrep¶

Will not solve.

In [157]:

;ipython nbconvert 1_pleac_string.ipynb

[NbConvertApp] Using existing profile dir: u'C:\\Users\\keithc\\.ipython\\profile_default'
[NbConvertApp] Converting notebook 1_pleac_string.ipynb to html
[NbConvertApp] Support files will be in 1_pleac_string_files\
[NbConvertApp] Loaded template html_full.tpl
[NbConvertApp] Writing 322023 bytes to 1_pleac_string.html

In [ ]: