File & Disk Tools#

Remove Things#

remove all comments from file: sed -e '/^#/d' <input_file> > <output_file>
remove all blank lines from file: awk '!/^$/' <input_file> > <output_file>

Split a File#

This will not split single lines:

split --line-bytes=<size> <file_to_split>

The size argument is an integer and optional unit (example: 10K is 10*1024). Units are K,M,G,T,P,E,Z,Y (powers of 1024) or KB,MB,… (powers of 1000).

Un-Split a File#

cat <file_to_split>-part-* > <file_to_split>

Remove Duplicates#

sort <file> | uniq -u > <new_file>

Delete Disk#

shred -vn 1 /dev/<device>

Checksums#

calculate checksum: sha512sum <file>
store checksums for multiple files: sha512sum * > sha512sum.txt
check checksums for multiple files: sha512sum -c sha512sum.txt