惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
Jina AI
Jina AI
Forbes - Security
Forbes - Security
雷峰网
雷峰网
人人都是产品经理
人人都是产品经理
博客园 - 叶小钗
V
Visual Studio Blog
月光博客
月光博客
博客园 - Franky
有赞技术团队
有赞技术团队
宝玉的分享
宝玉的分享
博客园 - 三生石上(FineUI控件)
酷 壳 – CoolShell
酷 壳 – CoolShell
Apple Machine Learning Research
Apple Machine Learning Research
The Register - Security
The Register - Security
S
SegmentFault 最新的问题
博客园 - 司徒正美
P
Proofpoint News Feed
Know Your Adversary
Know Your Adversary
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
A
Arctic Wolf
Cyberwarzone
Cyberwarzone
Simon Willison's Weblog
Simon Willison's Weblog
U
Unit 42
P
Proofpoint News Feed
Scott Helme
Scott Helme
MyScale Blog
MyScale Blog
T
Tenable Blog
Hugging Face - Blog
Hugging Face - Blog
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
小众软件
小众软件
C
CERT Recently Published Vulnerability Notes
P
Palo Alto Networks Blog
V
V2EX
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
T
Tailwind CSS Blog
V
Vulnerabilities – Threatpost
Latest news
Latest news
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
I
Intezer
Microsoft Azure Blog
Microsoft Azure Blog
爱范儿
爱范儿
博客园 - 【当耐特】
B
Blog RSS Feed
N
Netflix TechBlog - Medium
Recent Announcements
Recent Announcements
NISL@THU
NISL@THU
C
Cisco Blogs
C
CXSECURITY Database RSS Feed - CXSecurity.com
S
Schneier on Security

博客园 - FredGrit

C# JsonConvert DeserializeObject MissingMemberHandling.Ignore when the source model is completed and the required is partial Freezable objects do not require attachment to the WPF visual tree, maintain a persistent lifetime, and serve as a reliable binding relay between a detached ContextMenu and its parent host control. C# run httplistener to act as service application asynchronously in console, semaphoreslim allow the max concurrent number WPF Microsoft.Xaml.Behaviors.WPF, EventTrigger EventName="PreviewMouseDown" the tunnel event, while the MouseDown can't trigger the command because it was swallowed WPF customize command implemented ICommand, volatile read method is thread safe, preventing cpu and comipler reorder and optimization. WPF ItemsControl load huge 50M+ data WPF consume data generated by WCF periodically in json format WPF customize command based on ICommand and manually trigger WPF consume data generated by grpc services C# produce and consume data via Google.Protobuf WCF produce message and WPF consume periodically via DispatcherTimer WCF deconstruct WebConfig includes bindings, behaviors, service, endpoint ,serviceHostingEnvironment WPF SQLite SQLiteStudio WPF customize MultiSelectComboBox based on combobox WPF DataGrid Context menu binding command and commandparameter to datacontext WCF set fixed port as http://localhost:8888/ via Project /Properties/web/project url to create virtual directory WPF customize datagrid behavior based on behavior<datagrid> with command and command parameter WPF Microsoft Visual Studio XAML designer is busy WCF WebHttpBinding support both http and https WCF support basicHttpBinding and webHttpBinding - FredGrit WCF TestClient set fixed configuration file WPF consume http json and update periodically via DispatcherTimer WPF Prism.Core version 9.0.537 implemented navigation register singleton with splash screen, pass global variable via RegisterSingleton method WPF render periodically via DispatcherTimer, customize behavior - FredGrit Python cosume WCF service via requests in json format WPF call webHttpBinding from WCF WCF binding webHttpBinding is used to web browser in json format both in request and response WPF invoke WCF dll periodically via DispatcherTimer WCF webHttpBinding is open for web browser and wpf WPF DataTemplateSelector WPF DataGrid customize behavior with multiple commands and command parameters then invoke in mvvm - FredGrit WPF DataGrid behavior customize command and command parameter then invoke and implemented in MVVM - FredGrit WPF ItemsControl customize behavior and save all items WCF service can be accessed by browser WPF WCF produce data as service and WPF consume data as client periodically WPF GRPC and Probuf generated data as service then consume by wpf periodically WPF customize behavior based on Microsoft.Xaml.Behaviors.Wpf with command and commandparameter WPF call data from CPP wrapper dll via CLI\CLR - FredGrit WPF customize behavior WPF get gpu information via System.Management WPF ItemsControl IsItemsHost=True WPF Customize behavior and dependency property command C# Serilog, Serilog.Sinks.Console, Serilog.Sinks.File C# Serilog both in file and console Windows powershell view huge file via command C# serialize huge data more than 100M via splitting into batch and concatenating as one big json file WPF WeakReference C# serialize datetime then deserialize, print lose precision. resolve by ToString("o") C# produce data and send via WebSocket as service, Python,Flask,HTML as consumer invoke periodically C# write generated data service and sent via websocket, then consume by python periodically C# DateTime print precision to microseconds C# WebSocket console as service provide data, another console as client,send request periodically C# WebAPI [HttpGet("{cnt}"] pass argument WPF implement ICommand with async execute WPF ListBox control virtualization in mvvm WPF Data Source invoke from web api C# WebAPI
C# insert data into SQLite in batch periodically
FredGrit · 2026-05-21 · via 博客园 - FredGrit
Install-Package Microsoft.Data.Sqlite.Core
Install-Package SQLitePCLRaw.bundle_e_sqlite3
using Microsoft.Data.Sqlite;
using System.Runtime.Serialization;
using System.Text;
using Timer = System.Timers.Timer;

namespace ConsoleApp17
{
    internal class Program
    {
        static SqliteConnection sqliteConn;
        static System.Timers.Timer tmr;
        static List<Book> booksList;
        static long idx = 0;
        static void Main(string[] args)
        {
            OpenSqlConn();
            GenerateAndInsertData();
            tmr = new Timer();
            tmr.Elapsed += Tmr_Elapsed;
            tmr.Interval = 10000;
            tmr.Start();
            Console.ReadLine();
        }

        static long GetIncrementIdx()
        {
            return Interlocked.Increment(ref idx);
        }

        private static void Tmr_Elapsed(object? sender, System.Timers.ElapsedEventArgs e)
        {
            GenerateAndInsertData();
        }

        private static void GenerateAndInsertData()
        {
            Task.Run(() =>
            {
                InitBooksList(1000000);
                if (booksList != null && booksList.Any())
                {
                    string dbName = $"DB_{DateTime.Now.ToString("yyyyMM")}";
                    string tableName = $"Book_{DateTime.Now.ToString("yyyyMMdd")}";
                    CreateTableIfNotExists(dbName, tableName);
                    InsertIntoTableInBatch(tableName, booksList, 100000);
                }
            });           
        }

        private static void InsertIntoTableInBatch(string tableName, List<Book> booksList, int batchSize = 10000)
        {
            int booksCnt = booksList.Count;
            int batches = (booksCnt + batchSize - 1) / batchSize;
            for (int i = 0; i < batches; i++)
            {
                int start_idx = i * batchSize;
                int end_idx = Math.Min((i + 1) * batchSize, booksCnt);
                var batchBooks = booksList.Skip(start_idx).Take(batchSize).ToList();
                StringBuilder insertBuilder = new StringBuilder();
                insertBuilder.Append($"insert into {tableName} (name,isbn,author,abstract,content,comment,summary,title,topic) values ");

                foreach (var bk in batchBooks)
                {
                    insertBuilder.Append($"('{bk.Name}','{bk.ISBN}','{bk.Author}','{bk.Abstract}','{bk.Content}','{bk.Comment}','{bk.Summary}','{bk.Title}','{bk.Topic}'),");
                }
                string insertSQL = insertBuilder.ToString();
                insertSQL = insertSQL.Substring(0, insertSQL.Length - 1);
                ExecuteSQL(insertSQL);
                Console.WriteLine($"{DateTime.Now},Insert between First Id:{batchBooks.FirstOrDefault()?.Id} and Last Id:{batchBooks.LastOrDefault()?.Id} into {tableName} successfully");
            }
            Console.WriteLine($"{DateTime.Now},insert into {tableName} totally {booksCnt} items\n\n\n");
        }

        private static void CreateTableIfNotExists(string dbName = "DB_202605", string tableName = "Book_20260521")
        {
            string createTableSQL = $"create table if not exists {tableName} (id integer primary key autoincrement," +
                  "name varchar(100) not null default '',ISBN varchar(100) not null default '',Author varchar(100) not null default '',"
                  + "Abstract varchar(100) not null default '',Content varchar(100) not null default '',Comment varchar(100) not null default '',"
              + "Summary varchar(100) not null default '',Title varchar(100) not null default '',Topic varchar(100) not null default '')";
            ExecuteSQL(createTableSQL);           
        }

        private static void ExecuteSQL(string sql)
        {
            using (SqliteCommand cmd = new SqliteCommand(sql, sqliteConn))
            {
                cmd.ExecuteNonQuery();
            }
        }

        private static void InitBooksList(int cnt)
        {
            booksList = new List<Book>();
            for (int i = 0; i < cnt; i++)
            {
                var a = GetIncrementIdx();
                booksList.Add(new Book()
                {
                    Id = a,
                    Name = $"Name_{a}",
                    ISBN = $"ISBN_{a}_{Guid.NewGuid():N}",
                    Author = $"Author_{a}",
                    Abstract = $"Abstract_{a}",
                    Comment = $"Comment_{a}",
                    Content = $"Content_{a}",
                    Summary = $"Summary_{a}",
                    Title = $"Title_{a}",
                    Topic = $"Topic_{a}"
                });
            }
        }

        static void OpenSqlConn(string dbName = "")
        {
            if (string.IsNullOrWhiteSpace(dbName))
            {
                string connStr = $"Data source={DateTime.Now.ToString("yyyyMM")}.db";
                sqliteConn = new SqliteConnection(connStr);
                sqliteConn.Open();
            }
        }
    }

    [DataContract]
    public class Book
    {
        [DataMember]
        public long Id { get; set; }
        [DataMember]
        public string Name { get; set; }
        [DataMember]
        public string ISBN { get; set; }
        [DataMember]
        public string Author { get; set; }
        [DataMember]
        public string Abstract { get; set; }
        [DataMember]
        public string Comment { get; set; }
        [DataMember]
        public string Content { get; set; }
        [DataMember]
        public string Summary { get; set; }
        [DataMember]
        public string Title { get; set; }
        [DataMember]
        public string Topic { get; set; }
    }
}
using Microsoft.Data.Sqlite;
using System.Collections.Concurrent;
using System.Runtime.Serialization;
using System.Text;
using Timer = System.Timers.Timer;

namespace ConsoleApp17
{
    internal class Program
    {
        static System.Timers.Timer tmr;
        static long idx = 0;
        static bool _isProcessing = false;

        static void Main(string[] args)
        {
            SQLitePCL.Batteries.Init();
            GenerateAndInsertData();
            tmr = new Timer();
            tmr.Elapsed += Tmr_Elapsed;
            tmr.Interval = 10000;
            tmr.Start();
            Console.ReadLine();
        }

        static long GetIncrementIdx()
        {
            return Interlocked.Increment(ref idx);
        }

        private static void Tmr_Elapsed(object? sender, System.Timers.ElapsedEventArgs e)
        {
            if (_isProcessing)
            {
                return;
            }

            _isProcessing = true;

            try
            {
                GenerateAndInsertData();
            }
            catch (Exception ex)
            {
                Console.WriteLine($"{DateTime.Now},{ex.Message}");
            }
            finally
            {
                _isProcessing = false;
            }
        }

        private static void GenerateAndInsertData()
        {
            var now = DateTime.Now;
            string dbName = $"DB_{now:yyyyMM}";
            string tableName = $"Book_{now:yyyyMMdd}";
            string dbPath = $"{dbName}.db";
            using var conn = new SqliteConnection($"Data Source={dbPath}");
            conn.Open();

            CreateTableIfNotExists(conn, tableName);
            var books = GenerateBooks(1000000);
            BatchInsert(conn, tableName, books);
        }

        static List<Book> GenerateBooks(int count)
        {
            var list = new List<Book>(count);
            for (int i = 0; i < count; i++)
            {
                long id = GetIncrementIdx();
                list.Add(new Book
                {
                    Id = id,
                    Name = $"Name_{id}",
                    ISBN = $"ISBN_{id}_{Guid.NewGuid():N}",
                    Author = $"Author_{id}",
                    Abstract = $"Abstract_{id}",
                    Comment = $"Comment_{id}",
                    Content = $"Content_{id}",
                    Summary = $"Summary_{id}",
                    Title = $"Title_{id}",
                    Topic = $"Topic_{id}"
                });
            }
            return list;
        }

        static void CreateTableIfNotExists(SqliteConnection conn, string tableName)
        {
            string sql = $@"
                CREATE TABLE IF NOT EXISTS {tableName} (
                    id INTEGER PRIMARY KEY AUTOINCREMENT,
                    name TEXT NOT NULL DEFAULT '',
                    ISBN TEXT NOT NULL DEFAULT '',
                    Author TEXT NOT NULL DEFAULT '',
                    Abstract TEXT NOT NULL DEFAULT '',
                    Content TEXT NOT NULL DEFAULT '',
                    Comment TEXT NOT NULL DEFAULT '',
                    Summary TEXT NOT NULL DEFAULT '',
                    Title TEXT NOT NULL DEFAULT '',
                    Topic TEXT NOT NULL DEFAULT ''
                )";

            using var cmd = new SqliteCommand(sql, conn);
            cmd.ExecuteNonQuery();
        }
        
        static void BatchInsert(SqliteConnection conn, string tableName, List<Book> books)
        {
            using var trans = conn.BeginTransaction();

            try
            {                
                string insertSql = $@"
                    INSERT INTO {tableName} 
                    (name,ISBN,Author,Abstract,Content,Comment,Summary,Title,Topic) 
                    VALUES 
                    (@Name,@ISBN,@Author,@Abstract,@Content,@Comment,@Summary,@Title,@Topic)";

                using var cmd = new SqliteCommand(insertSql, conn, trans);
               
                cmd.Parameters.Add("@Name", SqliteType.Text);
                cmd.Parameters.Add("@ISBN", SqliteType.Text);
                cmd.Parameters.Add("@Author", SqliteType.Text);
                cmd.Parameters.Add("@Abstract", SqliteType.Text);
                cmd.Parameters.Add("@Content", SqliteType.Text);
                cmd.Parameters.Add("@Comment", SqliteType.Text);
                cmd.Parameters.Add("@Summary", SqliteType.Text);
                cmd.Parameters.Add("@Title", SqliteType.Text);
                cmd.Parameters.Add("@Topic", SqliteType.Text);

                foreach (var book in books)
                {
                    cmd.Parameters["@Name"].Value = book.Name;
                    cmd.Parameters["@ISBN"].Value = book.ISBN;
                    cmd.Parameters["@Author"].Value = book.Author;
                    cmd.Parameters["@Abstract"].Value = book.Abstract;
                    cmd.Parameters["@Content"].Value = book.Content;
                    cmd.Parameters["@Comment"].Value = book.Comment;
                    cmd.Parameters["@Summary"].Value = book.Summary;
                    cmd.Parameters["@Title"].Value = book.Title;
                    cmd.Parameters["@Topic"].Value = book.Topic;

                    cmd.ExecuteNonQuery();
                }

                trans.Commit();
                Console.WriteLine($"{DateTime.Now},insert {books.Count()} items into table {tableName}\n\n\n");
            }
            catch
            {
                trans.Rollback();
                throw;
            }
        }
    }

    [DataContract]
    public class Book
    {
        [DataMember] public long Id { get; set; }
        [DataMember] public string Name { get; set; }
        [DataMember] public string ISBN { get; set; }
        [DataMember] public string Author { get; set; }
        [DataMember] public string Abstract { get; set; }
        [DataMember] public string Comment { get; set; }
        [DataMember] public string Content { get; set; }
        [DataMember] public string Summary { get; set; }
        [DataMember] public string Title { get; set; }
        [DataMember] public string Topic { get; set; }
    }
}