Linux : How to split a big file in multiple small files

 

If you are splitting a Text file and want to split it by lines you can do this:

split -l 1000 allRecords.txt new

Which will split the text file in output files of 1000 lines each. This is another way to split a file and is mostly used for text files like logs, sql dumps, csv files, etc..

➜ prodFixAuditLog wc allRecords
901 914 66072 allRecords
➜ split -l 100 allRecords new
➜  ls -l  (will list multiple file with name new)
allRecords newac newaf newai
newaa newad newag newaj
newab newae newah

If you want to combine them back you can use below command

➜ cat new* > combineAllRecords.txt
This will combine all files starting with new into a big file with name combineAllRecords.txt

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s