10,000 scraps of paper

Walk the line…endings

Have a file where you want to add a comma to the end of each line?

$ sed 's/$/,/' ip_addresses.txt | head -n2

This should work…but, wait, it didn’t. Instead, it replaced the first character of each line with a comma. Why?

,6.127.0.0/17
,7.238.80.0/21

Aha, it is because the file is dos encoded. See the “\r \n”?

$ head -n1 ip_addresses.txt | od -c
0000000 9 6 . 1 2 7 . 0 . 0 / 1 7 \r \n
0000017

Ok, so dos2unix comes to the rescue.

$ dos2unix ip_addresses.txt
dos2unix: converting file ip_addresses.txt to Unix format...

And, finally, we have…

$ sed 's/$/,/' ip_addresses.txt | head -n2
96.127.0.0/17,
87.238.80.0/21,

Wait, not so fast. We actually wanted commas on all BUT the last line and observing the last two lines, we’ve got an extra comma.

$ sed 's/$/,/' ip_addresses.txt | tail -n2
103.246.150.0/23,
103.246.148.0/23,

So, use “$!” in sed which matches lines which are NOT the last line.

$ sed '$!s/$/,/' ip_addresses.txt | tail -n2
103.246.150.0/23,
103.246.148.0/2