Linux : How to split a big file in multiple small files


If you are splitting a Text file and want to split it by lines you can do this:

split -l 1000 allRecords.txt new

Which will split the text file in output files of 1000 lines each. This is another way to split a file and is mostly used for text files like logs, sql dumps, csv files, etc..

➜ prodFixAuditLog wc allRecords
901 914 66072 allRecords
➜ split -l 100 allRecords new
➜  ls -l  (will list multiple file with name new)
allRecords newac newaf newai
newaa newad newag newaj
newab newae newah

If you want to combine them back you can use below command

➜ cat new* > combineAllRecords.txt
This will combine all files starting with new into a big file with name combineAllRecords.txt