惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

GbyAI
GbyAI
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
C
Check Point Blog
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
Exploit-DB.com RSS Feed
Exploit-DB.com RSS Feed
J
Java Code Geeks
博客园 - 【当耐特】
H
Hacker News: Front Page
S
Secure Thoughts
博客园_首页
Engineering at Meta
Engineering at Meta
N
News | PayPal Newsroom
美团技术团队
SecWiki News
SecWiki News
U
Unit 42
The Hacker News
The Hacker News
有赞技术团队
有赞技术团队
T
The Exploit Database - CXSecurity.com
M
MIT News - Artificial intelligence
T
Threat Research - Cisco Blogs
V
Vulnerabilities – Threatpost
TaoSecurity Blog
TaoSecurity Blog
The Last Watchdog
The Last Watchdog
www.infosecurity-magazine.com
www.infosecurity-magazine.com
F
Fortinet All Blogs
T
Tor Project blog
T
Tailwind CSS Blog
Scott Helme
Scott Helme
Recorded Future
Recorded Future
Know Your Adversary
Know Your Adversary
The Register - Security
The Register - Security
Google DeepMind News
Google DeepMind News
Microsoft Security Blog
Microsoft Security Blog
D
Darknet – Hacking Tools, Hacker News & Cyber Security
Google DeepMind News
Google DeepMind News
S
Security @ Cisco Blogs
S
SegmentFault 最新的问题
Apple Machine Learning Research
Apple Machine Learning Research
月光博客
月光博客
阮一峰的网络日志
阮一峰的网络日志
H
Heimdal Security Blog
MongoDB | Blog
MongoDB | Blog
S
Securelist
C
CXSECURITY Database RSS Feed - CXSecurity.com
雷峰网
雷峰网
博客园 - 聂微东
S
Schneier on Security
T
Tenable Blog
C
Cyber Attacks, Cyber Crime and Cyber Security
B
Blog

博客园 - BloggerSb

Swagger 文档设置api版本 .Net Core Routing Demo .net Core读取配置比较 简化版DbExecutor,将DataTable映射到T属性(支持Dapper风格的匿名参数)。(编程题) 大文件单词统计 (编程题) ASP.NET Core CRUD API 创建 UserController,实现 Get, Post, Put, Delete 方法,使用 EF Core 访问数据库。 (编程题) 设计模式落地:Repository + UnitOfWork + CQRS 完整实现 (编程题) 实现一个带 CorrelationId、请求日志、异常统一处理的中间件链 (编程题) 异步限流器实现(编程题) 编程题,记录所有接口的执行耗时 .net面试题目 (问答题) 面试高频简答题 Aspose最新Slides破解 HttpContext.User.Identity.IsAuthenticated 为false 关于Cannot resolve scoped service from root provider解决方案 MongoDB用户权限管理,设置密码并连接 mongodb连接字符串 mongodb 使用 MongoDB Compass 创建账号,角色 安装mongodb bootstrap popover 设置悬浮框宽度 div contenteditable="true" 添加placehoder效果 光标自动定位到起始位置contenteditable="true" ,v-html绑定内容,div可编辑时,光标移到最前面
给定百万级订单表,实现高效分页 + 动态条件查询 + 导出 Excel(避免内存爆炸) (编程题)
BloggerSb · 2026-04-17 · via 博客园 - BloggerSb

EF Core 性能优化题 题:给定百万级订单表,实现高效分页 + 动态条件查询 + 导出 Excel(避免内存爆炸)。 关键:`AsNoTracking()` + `Skip/Take` + `IQueryable` 动态构建 + `ToListAsync` + NPOI 流式导出

// 订单实体
public class Order
{
    public long Id { get; set; }
    public string OrderNo { get; set; }
    public decimal Amount { get; set; }
    public DateTime CreateTime { get; set; }
    public int Status { get; set; } // 0=待支付 1=已完成 2=已取消
    public string CustomerName { get; set; }
}

// DbContext
public class AppDbContext : DbContext
{
    public DbSet<Order> Orders => Set<Order>();
    public AppDbContext(DbContextOptions<AppDbContext> options) : base(options) { }
}

// 订单查询条件
public class OrderQueryDto
{
    public string? OrderNo { get; set; }
    public DateTime? StartTime { get; set; }
    public DateTime? EndTime { get; set; }
    public int? Status { get; set; }
    public string? CustomerName { get; set; }
    
    // 分页参数
    public int PageIndex { get; set; } = 1;
    public int PageSize { get; set; } = 20;
}

•  AsNoTracking():禁用跟踪,大幅提升查询速度,减少内存占用 
•  IQueryable 动态构建:只拼接条件,不执行 SQL,最后统一执行 
•  Skip/Take:数据库端分页,只查当前页数据 
•  ToListAsync():异步执行,不阻塞线程,适配高并发
using Microsoft.EntityFrameworkCore;

public class OrderService
{
    private readonly AppDbContext _dbContext;
    public OrderService(AppDbContext dbContext)
    {
        _dbContext = dbContext;
    }

    /// <summary>
    /// 动态条件 + 高效分页查询
    /// </summary>
    public async Task<(List<Order> List, long TotalCount)> GetOrderPageAsync(OrderQueryDto query)
    {
        // 1. 基础查询:禁用跟踪 + 无锁查询(百万级必加)
        var queryable = _dbContext.Orders
            .AsNoTracking()
            .AsSplitQuery()  // 复杂查询拆分,避免笛卡尔积
            .AsQueryable();

        // 2. 动态构建条件(IQueryable 延迟执行,不触发查询)
        if (!string.IsNullOrWhiteSpace(query.OrderNo))
            queryable = queryable.Where(o => o.OrderNo.Contains(query.OrderNo));

        if (query.StartTime.HasValue)
            queryable = queryable.Where(o => o.CreateTime >= query.StartTime.Value);

        if (query.EndTime.HasValue)
            queryable = queryable.Where(o => o.CreateTime <= query.EndTime.Value);

        if (query.Status.HasValue)
            queryable = queryable.Where(o => o.Status == query.Status.Value);

        if (!string.IsNullOrWhiteSpace(query.CustomerName))
            queryable = queryable.Where(o => o.CustomerName.Contains(query.CustomerName));

        // 3. 获取总条数(仅 COUNT,高效)
        var totalCount = await queryable.LongCountAsync();

        // 4. 数据库端分页(核心:Skip/Take 生成 SQL 分页)
        var skip = (query.PageIndex - 1) * query.PageSize;
        var list = await queryable
            .OrderByDescending(o => o.CreateTime) // 必须排序,Skip/Take 要求
            .Skip(skip)
            .Take(query.PageSize)
            .ToListAsync();

        return (list, totalCount);
    }
}
•  不加载全表到内存:流式读取数据、流式写入 Excel 
•  NPOI 流式写入:SXSSF 模式,自动刷盘,内存占用极低 
•  分批次读取:每次只查 1000 条,循环写入,内存恒定
Install-Package NPOI
Install-Package NPOI.OpenXml4Net
Install-Package NPOI.OpenXmlFormats
using NPOI.SS.UserModel;
using NPOI.XSSF.Streaming;
using System.IO;

/// <summary>
/// 百万级订单流式导出 Excel(无内存爆炸)
/// </summary>
public async Task<byte[]> ExportOrderExcelAsync(OrderQueryDto query)
{
    // 1. 复用动态查询条件(不加载全表)
    var queryable = _dbContext.Orders
        .AsNoTracking()
        .AsQueryable();

    // 动态条件(和分页查询完全一致)
    if (!string.IsNullOrWhiteSpace(query.OrderNo))
        queryable = queryable.Where(o => o.OrderNo.Contains(query.OrderNo));
    if (query.StartTime.HasValue)
        queryable = queryable.Where(o => o.CreateTime >= query.StartTime.Value);
    if (query.EndTime.HasValue)
        queryable = queryable.Where(o => o.CreateTime <= query.EndTime.Value);
    if (query.Status.HasValue)
        queryable = queryable.Where(o => o.Status == query.Status.Value);
    if (!string.IsNullOrWhiteSpace(query.CustomerName))
        queryable = queryable.Where(o => o.CustomerName.Contains(query.CustomerName));

    // 2. 排序(必须)
    queryable = queryable.OrderByDescending(o => o.Id);

    // 3. NPOI 流式工作簿(SXSSF 关键:低内存模式)
    var workbook = new SXSSFWorkbook(1000); // 每次保留1000行在内存
    var sheet = workbook.CreateSheet("订单数据");

    // 4. 创建表头
    var headerRow = sheet.CreateRow(0);
    headerRow.CreateCell(0).SetCellValue("订单ID");
    headerRow.CreateCell(1).SetCellValue("订单号");
    headerRow.CreateCell(2).SetCellValue("订单金额");
    headerRow.CreateCell(3).SetCellValue("创建时间");
    headerRow.CreateCell(4).SetCellValue("状态");
    headerRow.CreateCell(5).SetCellValue("客户姓名");

    var rowIndex = 1;
    var batchSize = 1000; // 每批次读取1000条(可调整)
    long skip = 0;

    // 5. 流式循环读取 + 写入(核心:永不加载全表)
    while (true)
    {
        // 每次只查 1000 条
        var batchData = await queryable
            .Skip((int)skip)
            .Take(batchSize)
            .ToListAsync();

        if (batchData.Count == 0) break; // 无数据退出

        // 写入当前批次
        foreach (var order in batchData)
        {
            var row = sheet.CreateRow(rowIndex++);
            row.CreateCell(0).SetCellValue(order.Id);
            row.CreateCell(1).SetCellValue(order.OrderNo);
            row.CreateCell(2).SetCellValue((double)order.Amount);
            row.CreateCell(3).SetCellValue(order.CreateTime.ToString("yyyy-MM-dd HH:mm:ss"));
            row.CreateCell(4).SetCellValue(order.Status switch { 0 => "待支付", 1 => "已完成", 2 => "已取消" });
            row.CreateCell(5).SetCellValue(order.CustomerName);
        }

        skip += batchSize;
    }

    // 6. 流式写入到字节数组(不占用大内存)
    using var ms = new MemoryStream();
    workbook.Write(ms, true); // 流式写入
    workbook.Close();
    ms.Position = 0;

    return ms.ToArray();
}

最终 API 调用(Web API)
[ApiController]
[Route("api/orders")]
public class OrderController : ControllerBase
{
    private readonly OrderService _orderService;
    public OrderController(OrderService orderService)
    {
        _orderService = orderService;
    }

    // 分页查询
    [HttpGet("page")]
    public async Task<IActionResult> GetPage([FromQuery] OrderQueryDto query)
    {
        var (list, total) = await _orderService.GetOrderPageAsync(query);
        return Ok(new { List = list, Total = total });
    }

    // 流式导出
    [HttpGet("export")]
    public async Task<IActionResult> Export([FromQuery] OrderQueryDto query)
    {
        var bytes = await _orderService.ExportOrderExcelAsync(query);
        return File(bytes, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", 
            $"订单导出_{DateTime.Now:yyyyMMddHHmmss}.xlsx");
    }
}

数据库索引(性能提升 10~100 倍)

CREATE INDEX IX_Orders_CreateTime ON Orders(CreateTime);
CREATE INDEX IX_Orders_Status ON Orders(Status);
CREATE INDEX IX_Orders_OrderNo ON Orders(OrderNo);