Linux : How to split a big file in multiple small files


If you are splitting a Text file and want to split it by lines you can do this:

split -l 1000 allRecords.txt new

Which will split the text file in output files of 1000 lines each. This is another way to split a file and is mostly used for text files like logs, sql dumps, csv files, etc..

➜ prodFixAuditLog wc allRecords
901 914 66072 allRecords
➜ split -l 100 allRecords new
➜  ls -l  (will list multiple file with name new)
allRecords newac newaf newai
newaa newad newag newaj
newab newae newah

If you want to combine them back you can use below command

➜ cat new* > combineAllRecords.txt
This will combine all files starting with new into a big file with name combineAllRecords.txt

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.