惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Engineering at Meta
Engineering at Meta
Microsoft Azure Blog
Microsoft Azure Blog
I
InfoQ
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
人人都是产品经理
人人都是产品经理
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
T
Tailwind CSS Blog
MongoDB | Blog
MongoDB | Blog
Google DeepMind News
Google DeepMind News
WordPress大学
WordPress大学
量子位
美团技术团队
大猫的无限游戏
大猫的无限游戏
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Last Week in AI
Last Week in AI
博客园 - 司徒正美
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
小众软件
小众软件
C
Check Point Blog
博客园 - 三生石上(FineUI控件)
N
Netflix TechBlog - Medium
Recent Announcements
Recent Announcements
有赞技术团队
有赞技术团队
月光博客
月光博客

博客园 - FredGrit

WPF customize via three combied custom controls WPF DataGrid DataGridTemplateColumn DataTemplate ContentPresenter ContentTemplate WPF Customcontrol NumUpDownScroller WPF CustomControl override Template in Generic.xaml and invoke different style WPF HierarchicalDataTemplate customize via ToggleButton WPF DataGrid DataGridTemplateColumn DataTemplate call predefined DataTemplate via ContentPresenter and ContentTemplate WPF ListBox load data via contentcontrol an ItemTemplate WPF DataGrid load data from Asp.Net Core WebAPI WPF TreeView HierarchicalDataTemplate with grouped Data WPF display grouped data with GroupBox and ItemsControl WPF two listbox scroll syncchronously via behavior WPF invoke data from WebAPI,DataGridTemplate call pre defined DataTemplate via ContentPresenter WPF ListBox load data from WCF WPF datagrid load data from WCF via json, export selected items to json file WPF ItemsControl load data from WCF,DataTemplate, ContentControl WPF customize rotated wheel relentlessly via custom control WPF embed DataTemplate in HierchicalDataTemplate of TreeView WPF ContentControl, ItemsControl, ItemsPanelTemplate,VirtualizingStackPanel,convert xml string to List<T> WPF parse web.config recursively, TreeView and HierarchicalDataTemplate WPF Custom control in cs and Generic.xaml WPF ContextMenu independent visual tree resolved via Freezable implemented class WPF custom control GetTemplateChild vs FindName,NameScope separation between Logical Tree and Visual Tree, WPF ListBox ListView Datatemplate, parse xml to List via XmlSerializer and StringReader WPF TreeView HierarchicalDataTemplate, parse xml via traverse in XmlElement WPF parse xml via [XmRoot] and [XmlElement] attributes together, contentcontrol's template is ControlTemplate from Resources WPF ContentControl invoke Template from resource, reuse datatemplate WPF deserialize xml string as List via [XmlRoot] and [XmlElement] attribute WPF ContentControl Template WPF DatagridTemplate Binding DataTemplate WPF TreeView explicitly given key name to HierarchicalDataTemplate
C# insert data into SQLite in batch periodically
FredGrit · 2026-05-21 · via 博客园 - FredGrit
Install-Package Microsoft.Data.Sqlite.Core
Install-Package SQLitePCLRaw.bundle_e_sqlite3
using Microsoft.Data.Sqlite;
using System.Runtime.Serialization;
using System.Text;
using Timer = System.Timers.Timer;

namespace ConsoleApp17
{
    internal class Program
    {
        static SqliteConnection sqliteConn;
        static System.Timers.Timer tmr;
        static List<Book> booksList;
        static long idx = 0;
        static void Main(string[] args)
        {
            OpenSqlConn();
            GenerateAndInsertData();
            tmr = new Timer();
            tmr.Elapsed += Tmr_Elapsed;
            tmr.Interval = 10000;
            tmr.Start();
            Console.ReadLine();
        }

        static long GetIncrementIdx()
        {
            return Interlocked.Increment(ref idx);
        }

        private static void Tmr_Elapsed(object? sender, System.Timers.ElapsedEventArgs e)
        {
            GenerateAndInsertData();
        }

        private static void GenerateAndInsertData()
        {
            Task.Run(() =>
            {
                InitBooksList(1000000);
                if (booksList != null && booksList.Any())
                {
                    string dbName = $"DB_{DateTime.Now.ToString("yyyyMM")}";
                    string tableName = $"Book_{DateTime.Now.ToString("yyyyMMdd")}";
                    CreateTableIfNotExists(dbName, tableName);
                    InsertIntoTableInBatch(tableName, booksList, 100000);
                }
            });           
        }

        private static void InsertIntoTableInBatch(string tableName, List<Book> booksList, int batchSize = 10000)
        {
            int booksCnt = booksList.Count;
            int batches = (booksCnt + batchSize - 1) / batchSize;
            for (int i = 0; i < batches; i++)
            {
                int start_idx = i * batchSize;
                int end_idx = Math.Min((i + 1) * batchSize, booksCnt);
                var batchBooks = booksList.Skip(start_idx).Take(batchSize).ToList();
                StringBuilder insertBuilder = new StringBuilder();
                insertBuilder.Append($"insert into {tableName} (name,isbn,author,abstract,content,comment,summary,title,topic) values ");

                foreach (var bk in batchBooks)
                {
                    insertBuilder.Append($"('{bk.Name}','{bk.ISBN}','{bk.Author}','{bk.Abstract}','{bk.Content}','{bk.Comment}','{bk.Summary}','{bk.Title}','{bk.Topic}'),");
                }
                string insertSQL = insertBuilder.ToString();
                insertSQL = insertSQL.Substring(0, insertSQL.Length - 1);
                ExecuteSQL(insertSQL);
                Console.WriteLine($"{DateTime.Now},Insert between First Id:{batchBooks.FirstOrDefault()?.Id} and Last Id:{batchBooks.LastOrDefault()?.Id} into {tableName} successfully");
            }
            Console.WriteLine($"{DateTime.Now},insert into {tableName} totally {booksCnt} items\n\n\n");
        }

        private static void CreateTableIfNotExists(string dbName = "DB_202605", string tableName = "Book_20260521")
        {
            string createTableSQL = $"create table if not exists {tableName} (id integer primary key autoincrement," +
                  "name varchar(100) not null default '',ISBN varchar(100) not null default '',Author varchar(100) not null default '',"
                  + "Abstract varchar(100) not null default '',Content varchar(100) not null default '',Comment varchar(100) not null default '',"
              + "Summary varchar(100) not null default '',Title varchar(100) not null default '',Topic varchar(100) not null default '')";
            ExecuteSQL(createTableSQL);           
        }

        private static void ExecuteSQL(string sql)
        {
            using (SqliteCommand cmd = new SqliteCommand(sql, sqliteConn))
            {
                cmd.ExecuteNonQuery();
            }
        }

        private static void InitBooksList(int cnt)
        {
            booksList = new List<Book>();
            for (int i = 0; i < cnt; i++)
            {
                var a = GetIncrementIdx();
                booksList.Add(new Book()
                {
                    Id = a,
                    Name = $"Name_{a}",
                    ISBN = $"ISBN_{a}_{Guid.NewGuid():N}",
                    Author = $"Author_{a}",
                    Abstract = $"Abstract_{a}",
                    Comment = $"Comment_{a}",
                    Content = $"Content_{a}",
                    Summary = $"Summary_{a}",
                    Title = $"Title_{a}",
                    Topic = $"Topic_{a}"
                });
            }
        }

        static void OpenSqlConn(string dbName = "")
        {
            if (string.IsNullOrWhiteSpace(dbName))
            {
                string connStr = $"Data source={DateTime.Now.ToString("yyyyMM")}.db";
                sqliteConn = new SqliteConnection(connStr);
                sqliteConn.Open();
            }
        }
    }

    [DataContract]
    public class Book
    {
        [DataMember]
        public long Id { get; set; }
        [DataMember]
        public string Name { get; set; }
        [DataMember]
        public string ISBN { get; set; }
        [DataMember]
        public string Author { get; set; }
        [DataMember]
        public string Abstract { get; set; }
        [DataMember]
        public string Comment { get; set; }
        [DataMember]
        public string Content { get; set; }
        [DataMember]
        public string Summary { get; set; }
        [DataMember]
        public string Title { get; set; }
        [DataMember]
        public string Topic { get; set; }
    }
}
using Microsoft.Data.Sqlite;
using System.Collections.Concurrent;
using System.Runtime.Serialization;
using System.Text;
using Timer = System.Timers.Timer;

namespace ConsoleApp17
{
    internal class Program
    {
        static System.Timers.Timer tmr;
        static long idx = 0;
        static bool _isProcessing = false;

        static void Main(string[] args)
        {
            SQLitePCL.Batteries.Init();
            GenerateAndInsertData();
            tmr = new Timer();
            tmr.Elapsed += Tmr_Elapsed;
            tmr.Interval = 10000;
            tmr.Start();
            Console.ReadLine();
        }

        static long GetIncrementIdx()
        {
            return Interlocked.Increment(ref idx);
        }

        private static void Tmr_Elapsed(object? sender, System.Timers.ElapsedEventArgs e)
        {
            if (_isProcessing)
            {
                return;
            }

            _isProcessing = true;

            try
            {
                GenerateAndInsertData();
            }
            catch (Exception ex)
            {
                Console.WriteLine($"{DateTime.Now},{ex.Message}");
            }
            finally
            {
                _isProcessing = false;
            }
        }

        private static void GenerateAndInsertData()
        {
            var now = DateTime.Now;
            string dbName = $"DB_{now:yyyyMM}";
            string tableName = $"Book_{now:yyyyMMdd}";
            string dbPath = $"{dbName}.db";
            using var conn = new SqliteConnection($"Data Source={dbPath}");
            conn.Open();

            CreateTableIfNotExists(conn, tableName);
            var books = GenerateBooks(1000000);
            BatchInsert(conn, tableName, books);
        }

        static List<Book> GenerateBooks(int count)
        {
            var list = new List<Book>(count);
            for (int i = 0; i < count; i++)
            {
                long id = GetIncrementIdx();
                list.Add(new Book
                {
                    Id = id,
                    Name = $"Name_{id}",
                    ISBN = $"ISBN_{id}_{Guid.NewGuid():N}",
                    Author = $"Author_{id}",
                    Abstract = $"Abstract_{id}",
                    Comment = $"Comment_{id}",
                    Content = $"Content_{id}",
                    Summary = $"Summary_{id}",
                    Title = $"Title_{id}",
                    Topic = $"Topic_{id}"
                });
            }
            return list;
        }

        static void CreateTableIfNotExists(SqliteConnection conn, string tableName)
        {
            string sql = $@"
                CREATE TABLE IF NOT EXISTS {tableName} (
                    id INTEGER PRIMARY KEY AUTOINCREMENT,
                    name TEXT NOT NULL DEFAULT '',
                    ISBN TEXT NOT NULL DEFAULT '',
                    Author TEXT NOT NULL DEFAULT '',
                    Abstract TEXT NOT NULL DEFAULT '',
                    Content TEXT NOT NULL DEFAULT '',
                    Comment TEXT NOT NULL DEFAULT '',
                    Summary TEXT NOT NULL DEFAULT '',
                    Title TEXT NOT NULL DEFAULT '',
                    Topic TEXT NOT NULL DEFAULT ''
                )";

            using var cmd = new SqliteCommand(sql, conn);
            cmd.ExecuteNonQuery();
        }
        
        static void BatchInsert(SqliteConnection conn, string tableName, List<Book> books)
        {
            using var trans = conn.BeginTransaction();

            try
            {                
                string insertSql = $@"
                    INSERT INTO {tableName} 
                    (name,ISBN,Author,Abstract,Content,Comment,Summary,Title,Topic) 
                    VALUES 
                    (@Name,@ISBN,@Author,@Abstract,@Content,@Comment,@Summary,@Title,@Topic)";

                using var cmd = new SqliteCommand(insertSql, conn, trans);
               
                cmd.Parameters.Add("@Name", SqliteType.Text);
                cmd.Parameters.Add("@ISBN", SqliteType.Text);
                cmd.Parameters.Add("@Author", SqliteType.Text);
                cmd.Parameters.Add("@Abstract", SqliteType.Text);
                cmd.Parameters.Add("@Content", SqliteType.Text);
                cmd.Parameters.Add("@Comment", SqliteType.Text);
                cmd.Parameters.Add("@Summary", SqliteType.Text);
                cmd.Parameters.Add("@Title", SqliteType.Text);
                cmd.Parameters.Add("@Topic", SqliteType.Text);

                foreach (var book in books)
                {
                    cmd.Parameters["@Name"].Value = book.Name;
                    cmd.Parameters["@ISBN"].Value = book.ISBN;
                    cmd.Parameters["@Author"].Value = book.Author;
                    cmd.Parameters["@Abstract"].Value = book.Abstract;
                    cmd.Parameters["@Content"].Value = book.Content;
                    cmd.Parameters["@Comment"].Value = book.Comment;
                    cmd.Parameters["@Summary"].Value = book.Summary;
                    cmd.Parameters["@Title"].Value = book.Title;
                    cmd.Parameters["@Topic"].Value = book.Topic;

                    cmd.ExecuteNonQuery();
                }

                trans.Commit();
                Console.WriteLine($"{DateTime.Now},insert {books.Count()} items into table {tableName}\n\n\n");
            }
            catch
            {
                trans.Rollback();
                throw;
            }
        }
    }

    [DataContract]
    public class Book
    {
        [DataMember] public long Id { get; set; }
        [DataMember] public string Name { get; set; }
        [DataMember] public string ISBN { get; set; }
        [DataMember] public string Author { get; set; }
        [DataMember] public string Abstract { get; set; }
        [DataMember] public string Comment { get; set; }
        [DataMember] public string Content { get; set; }
        [DataMember] public string Summary { get; set; }
        [DataMember] public string Title { get; set; }
        [DataMember] public string Topic { get; set; }
    }
}