惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

B
Blog RSS Feed
K
Kaspersky official blog
Forbes - Security
Forbes - Security
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
P
Proofpoint News Feed
G
GRAHAM CLULEY
V
Vulnerabilities – Threatpost
Security Latest
Security Latest
Scott Helme
Scott Helme
S
Securelist
美团技术团队
T
Threat Research - Cisco Blogs
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
S
SegmentFault 最新的问题
W
WeLiveSecurity
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
Apple Machine Learning Research
Apple Machine Learning Research
The Cloudflare Blog
AI
AI
L
Lohrmann on Cybersecurity
S
Security Affairs
Cloudbric
Cloudbric
SecWiki News
SecWiki News
爱范儿
爱范儿
雷峰网
雷峰网
Engineering at Meta
Engineering at Meta
C
Cyber Attacks, Cyber Crime and Cyber Security
大猫的无限游戏
大猫的无限游戏
N
News and Events Feed by Topic
I
InfoQ
S
Secure Thoughts
AWS News Blog
AWS News Blog
A
About on SuperTechFans
Schneier on Security
Schneier on Security
酷 壳 – CoolShell
酷 壳 – CoolShell
The Last Watchdog
The Last Watchdog
D
Darknet – Hacking Tools, Hacker News & Cyber Security
C
Check Point Blog
P
Palo Alto Networks Blog
博客园 - 【当耐特】
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
Google DeepMind News
Google DeepMind News
Latest news
Latest news
I
Intezer
博客园_首页
C
CXSECURITY Database RSS Feed - CXSecurity.com
V
V2EX
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
L
LangChain Blog
D
Docker

博客园 - 三叶草╮

DeepAgents 长期记忆相关组件详解 DeepAgents 长期记忆 笔记 Python 机器学习03 - 常见分类算法 Python 机器学习02 - 常见分类算法 Python uv 包管理 Python机器学习01 - Sklearn Python高级编程笔记 (线程/进程/协程) Python高级编程笔记 Python 基础笔记 Pandas 常用操作 (缺失值处理/排序/字符串处理/Index/Merge/合并) Python Pandas Python playwright 笔记 pipreqs:快速准确生成当前项目的requirements.txt,还有和freeze的对比 WPF 4款 UI 库 C# Selenium [转]在WPF中自定义控件 UserControl [转]WPF的依赖属性是怎么节约内存的 [转]WPF中的导航框架 [转]C#对Excel报表进行操作(读写和基本操作)
C# 模拟http请求网页数据 [网页爬虫]
三叶草╮ · 2018-02-27 · via 博客园 - 三叶草╮
using System;
using System.Collections.Specialized;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;
using FTE.Framework.Log4NetService;

namespace Proxy.BllServices
{
    /// <summary>
    /// http访问类
    /// </summary>
    public class HttpHelper
    {
        /// <summary>
        /// 访问失败的统一返回字符
        /// </summary>
        public String ErrorReturn { get; private set; } = "HttpHelper access error!";

        /// <summary>
        /// 登录后保存的cookie
        /// </summary>
        private CookieContainer Cookie = new CookieContainer();

        /// <summary>
        /// http post 访问网页
        /// </summary>
        /// <param name="Url"></param>
        /// <param name="postDataStr"></param>
        /// <returns></returns>
        public string HttpPostString(string Url, string postDataStr)
        {
            try
            {
                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(Url);
                request.Method = "POST";
                request.ContentType = "application/x-www-form-urlencoded";
                request.ContentLength = Encoding.UTF8.GetByteCount(postDataStr);
                request.CookieContainer = Cookie;
                Stream myRequestStream = request.GetRequestStream();
                StreamWriter myStreamWriter = new StreamWriter(myRequestStream, Encoding.GetEncoding("gb2312"));
                myStreamWriter.Write(postDataStr);
                myStreamWriter.Close();

                HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                response.Cookies = Cookie.GetCookies(response.ResponseUri);

                Stream myResponseStream = response.GetResponseStream();
                StreamReader myStreamReader = new StreamReader(myResponseStream, Encoding.GetEncoding("utf-8"));
                string retString = myStreamReader.ReadToEnd();
                myStreamReader.Close();
                myResponseStream.Close();

                return retString;
            }
            catch (Exception ex)
            {
                LoggerManagerSingle.Instance.Error("http post 网站出错", ex);
            }

            return ErrorReturn;
        }

        public string HttpGet(string Url, string postDataStr)
        {
            try
            {
                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(Url + (postDataStr == "" ? "" : "?") + postDataStr);
                request.Method = "GET";
                request.ContentType = "text/html;charset=UTF-8";
                request.CookieContainer = Cookie;

                HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                Stream myResponseStream = response.GetResponseStream();
                StreamReader myStreamReader = new StreamReader(myResponseStream, Encoding.GetEncoding("utf-8"));
                string retString = myStreamReader.ReadToEnd();
                myStreamReader.Close();
                myResponseStream.Close();

                return retString;
            }
            catch (Exception ex)
            {
                LoggerManagerSingle.Instance.Error("http get 网站出错", ex);
            }

            return ErrorReturn;
        }

        /// <summary>
        /// 使用form方式post数据[不包含文件]
        /// </summary>
        /// <param name="url"></param>
        /// <param name="stringDict"></param>
        /// <returns></returns>
        public string HttpPostForm(string url, NameValueCollection stringDict)
        {
            try
            {
                string responseContent;
                var memStream = new MemoryStream();
                var webRequest = (HttpWebRequest)WebRequest.Create(url);
                // 边界符  
                var boundary = "---------------" + DateTime.Now.Ticks.ToString("x");
                // 边界符  
                var beginBoundary = Encoding.ASCII.GetBytes("--" + boundary + "\r\n");
                // 最后的结束符  
                var endBoundary = Encoding.ASCII.GetBytes("--" + boundary + "--\r\n");

                // 设置属性  
                webRequest.CookieContainer = Cookie;
                webRequest.Method = "POST";
                webRequest.ContentType = "multipart/form-data; boundary=" + boundary;
                // 写入字符串的Key  
                var stringKeyHeader = "\r\n--" + boundary +
                                       "\r\nContent-Disposition: form-data; name=\"{0}\"" +
                                       "\r\n\r\n{1}\r\n";

                foreach (byte[] formitembytes in from string key in stringDict.Keys
                                                 select string.Format(stringKeyHeader, key, stringDict[key])
                                                     into formitem
                                                 select Encoding.UTF8.GetBytes(formitem))
                {
                    memStream.Write(formitembytes, 0, formitembytes.Length);
                }

                // 写入最后的结束边界符  
                memStream.Write(endBoundary, 0, endBoundary.Length);

                webRequest.ContentLength = memStream.Length;

                var requestStream = webRequest.GetRequestStream();

                memStream.Position = 0;
                var tempBuffer = new byte[memStream.Length];
                memStream.Read(tempBuffer, 0, tempBuffer.Length);
                memStream.Close();

                requestStream.Write(tempBuffer, 0, tempBuffer.Length);
                requestStream.Close();

                var httpWebResponse = (HttpWebResponse)webRequest.GetResponse();

                using (var httpStreamReader = new StreamReader(httpWebResponse.GetResponseStream(),
                                                                Encoding.GetEncoding("utf-8")))
                {
                    responseContent = httpStreamReader.ReadToEnd();
                }

                httpWebResponse.Close();
                webRequest.Abort();

                return responseContent;
            }
            catch (Exception ex)
            {
                LoggerManagerSingle.Instance.Error("http form post 网站出错", ex);
            }

            return ErrorReturn;
        }
    }
}

              调用例子:

//1.
helper.HttpPostString("http://192.168.1.1/", "luci_username=root&luci_password=password");

//2.
NameValueCollection stringDict = new NameValueCollection();
stringDict.Add("token", token);
stringDict.Add("cbid.wireless.default_radio1.ssid", "everTestWifi");
helper.HttpPostData("http://192.168.1.1/cgi-bin/luci/admin/network/wireless/radio1.network2", stringDict);

  参考连接:

    http://www.cnblogs.com/xssxss/archive/2012/07/03/2574554.html

    http://blog.csdn.net/flymorn/article/details/6769722