The method returns a file pointer as long as the file exists, otherwise it will return False in case of failure. It expects as second argument the mode in which we'll operate, in this case, just reading with the r identifier. To read the file, we will use the fopen function of PHP, this inbuilt function is used to simply open a file from a local URL, it's used to bind a resource to a steam. Implementing the read and iteration script You can of course do the same using another approach, but, you get the idea right? Split the file into smaller chunks that can be easily processed by your scripts later. We wrote an article previously of how to split huge CSV datasets into smaller chunks using CSV splitter, a tool for Windows 10. The smaller the file, the better will it be to have an optimal performance and control over your script, not only about the performance perspective but the logic as well.
So, the correct approach for such cases is not to work with the file directly, but with smaller files. To get started, when we talk about huge files, we are not talking about files with 50K or 70K rows, we talk about millions of rows like in this example, with a CSV file of 25GB.
In this article, we will explain you our approach for reading efficiently a huge CSV file in PHP. Obviously, because of the logic that the project required, we couldn't just simply import the file into the database through a tool like PHPMyAdmin, as every row in the CSV should be modificated to fit with our new database design. For example, in our company, we work with MySQL and our client came up with a CSV file of approximately 25GB with ~7.5M rows. If you work for a company that offers development services even for other companies of the same industry, you may have had this "wonderful" task of importing a huge "database" from a client into the database engine preferred by your company.
0 kommentar(er)
