php代码处理大量数据时需要注意的问题

php代码处理大量数据时需要注意的问题

内存超限,响应超时

遇到“Allowed memory size of 134217728 bytes exhausted (tried to allocate 16777224 bytes)”这类错误时，通常意味着你的程序尝试分配的内存超过了PHP配置中允许的最大内存限制。这种情况常见于使用PHP开发Web应用时，尤其是在处理大型数据或执行复杂运算时。

解决方案

动态设置

set_time_limit(0); //时间超时
ini_set('memory_limit', '-1'); //单文件脚本内存超出

处理大量数据时

SQLSTATE[HY000]: General error: 1390 Prepared statement contains too many placeholders
问题解析：它的意思是: 常规错误：1390预处理语句包含太多占位符. 说白了就是SQL语句太长了, 无法插入... >

解决方案

将整个数组分批插入，切记不要一条一条插入这样会给数据库带来压力,\

重点在于使用array_chunk(数据,条数)分割数据来处理.

提示

在laravel手册中也有chunkById和lazyById方法和说明

<?php
/*
 * @Descripttion:
 * @version: v1
 * @Author: youhujun 2900976495@qq.com
 * @Date: 2023-02-14 13:45:06
 * @LastEditors: youhujun 2900976495@qq.com
 * @LastEditTime: 2023-08-31 16:55:49
 * @FilePath: \api.laravel.com_LV9\database\seeders\Business\BusinessShowRegionUnionSeeder.php
 */

namespace Database\Seeders\Business;

use Illuminate\Database\Console\Seeds\WithoutModelEvents;
use Illuminate\Database\Seeder;
use Illuminate\Support\Facades\DB;

class BusinessShowRegionUnionSeeder extends Seeder
{
    /**
     * Run the database seeds.
     *
     * @return void
     */
    public function run()
    {

        $businessShowRegionUnionData = [];

        for ($i=1; $i <=5024 ; $i++)
        {
           $businessShowRegionUnionData[] =  ['show_config_id'=>1,'region_id'=>$i,'is_show'=>1,'created_time'=>time(),'created_at'=>date('Y-m-d H:i:s',time())];
           $businessShowRegionUnionData[] =  ['show_config_id'=>2,'region_id'=>$i,'is_show'=>1,'created_time'=>time(),'created_at'=>date('Y-m-d H:i:s',time())];
           $businessShowRegionUnionData[] =  ['show_config_id'=>3,'region_id'=>$i,'is_show'=>1,'created_time'=>time(),'created_at'=>date('Y-m-d H:i:s',time())];
        }

         $businessShowRegionUnionData[] =  ['show_config_id'=>1,'region_id'=>45055,'is_show'=>1,'created_time'=>time(),'created_at'=>date('Y-m-d H:i:s',time())];
         $businessShowRegionUnionData[] =  ['show_config_id'=>2,'region_id'=>45055,'is_show'=>1,'created_time'=>time(),'created_at'=>date('Y-m-d H:i:s',time())];
         $businessShowRegionUnionData[] =  ['show_config_id'=>3,'region_id'=>45055,'is_show'=>1,'created_time'=>time(),'created_at'=>date('Y-m-d H:i:s',time())];

        $chunk_list = array_chunk($businessShowRegionUnionData, 1000);

        foreach ($chunk_list as $new_list)
        {
            DB::table('business_show_region_union')->insert($new_list);
        }

    }
}