惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

T
Threat Research - Cisco Blogs
S
Securelist
H
Heimdal Security Blog
Scott Helme
Scott Helme
D
Darknet – Hacking Tools, Hacker News & Cyber Security
The Hacker News
The Hacker News
C
CXSECURITY Database RSS Feed - CXSecurity.com
Spread Privacy
Spread Privacy
Cyberwarzone
Cyberwarzone
V
Vulnerabilities – Threatpost
C
Cybersecurity and Infrastructure Security Agency CISA
C
CERT Recently Published Vulnerability Notes
P
Proofpoint News Feed
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
人人都是产品经理
人人都是产品经理
C
Cisco Blogs
www.infosecurity-magazine.com
www.infosecurity-magazine.com
Engineering at Meta
Engineering at Meta
Project Zero
Project Zero
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
有赞技术团队
有赞技术团队
T
Tailwind CSS Blog
Cisco Talos Blog
Cisco Talos Blog
Last Week in AI
Last Week in AI
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
O
OpenAI News
P
Proofpoint News Feed
Google Online Security Blog
Google Online Security Blog
Recent Announcements
Recent Announcements
Hacker News: Ask HN
Hacker News: Ask HN
美团技术团队
Stack Overflow Blog
Stack Overflow Blog
U
Unit 42
P
Privacy International News Feed
Google DeepMind News
Google DeepMind News
G
GRAHAM CLULEY
Apple Machine Learning Research
Apple Machine Learning Research
TaoSecurity Blog
TaoSecurity Blog
S
Security @ Cisco Blogs
C
Check Point Blog
H
Hackread – Cybersecurity News, Data Breaches, AI and More
Jina AI
Jina AI
S
Secure Thoughts
G
Google Developers Blog
C
Cyber Attacks, Cyber Crime and Cyber Security
L
LINUX DO - 最新话题
T
Tenable Blog
Latest news
Latest news
I
InfoQ

博客园 - 陋室

成功与一截树枝 sql 操作全集-整理收藏 JavaScript使用技巧精萃(转载) ASP.NET 中在指定的位置处插入字符 提示按下大写键的控件:MQTool(提供下载) 如何在mail的正文显示图片 NET设计模式之一:装饰模式(Decorator Pattern) 微软AJAX 教学系列第一讲:ScriptManager控件 微软AJAX 教学系列第一讲:局部刷新Partial Page Updates(翻译) 李开复:21世纪最需要的7种人才 超卓越的你_读完后让你恢复自信 how to write professional business letters? 到底什么是托管,什么是非托管的研究 数据库设计中的五个范式(本文转载,收藏下) 如何直接用XML做数据源? ASP.NET程序的优化建议<转> ASP.NET部署与安装_MSI制作图文教程. 在打包的时候,创建应用程序池,并自动将程序assign到新创建的池中(MSI制作) Understand SQL Cache Notifications
Building a File Service
陋室 · 2008-05-11 · via 博客园 - 陋室

Problem:

Managing files web applications should be quick and easy and most importantly, consistent. The traditional way to store files is on the file system or within a RDBMS (SQL SERVER, Oracle, MySql) or on SharePoint. However what if you were developing several applications that shared a million files and each was 5MB. Which solution would you choose? This is commonly the case for large content driven sites (Music Stores, video sharing sites, content management systems, etc).

Solution:

In providing a solution I would implement a SOA based File Service, and this article will guide you to making your own File Service. The reasons that I believe a File Service works are:

  • Single point of storage for data
  • Can be used by multiple applications
  • Single source for security
  • Easy to de-duplicate files
  • Clear separation between binary data and metadata
  • Efficient file system usage ensuring that director retrieval is fast
  • Efficient IO usage using multiple disks (per share)

Data Structure:

Firstly we need to create a table to store our file information. Most columns on the table shown in Figure 1 are self explanatory and for possible enhancements see the "Future Enhancements" section below.

Figure 1
Figure 1

Solution Structure:

  1. Create a new Solution in Visual Studio 2008, targeting the .NET Framework 3.5
  2. Create a new Class Library called "FileServiceCore"
    1. Add a new class called "FileHandlerFactory.cs"
    2. Add a new class called "FileHandlerLinq.cs"
    3. Add a new class called "FileDistributor.cs"
    4. Add a new LINQ to SQL Classes called "FileService.dbml"
    5. Using the Server Explorer create a new connection to the database that contains the "Files" table described above. You should be able to drag and drop the table onto the designer.
  3. Create a new Website called "FileService"
    1. Add a project reference to the FileServiceCore
    2. In the website root add a new generic handler called "ImageHandler.ashx"
    3. In the website root add a new generic handler called "DownloadHandler.ashx"
  4. Create a new website called "SampleWebsite"
    1. Add a project reference to the FileServiceCore

Your solution should now look like figure 2 below.

Figure 2
Figure 2

Logic:

Now that you have created your Data Structure and Visual Studio solution we will need to add the application logic to each of the classes (e.g. FileHandlerFactory.cs).

FileDistributor.cs

The FileDistributor class is responsible for distributing files to the file system. It contains a single static method that calculates a network file path for a FileId.

namespace FileServiceCore
{
public static class FileDistributor
{
/// <summary>
/// We use a static variable to tell the system which path to use
/// This ensures the file distributor acts as similar to a Round robin DNS
/// </summary>

private static volatile int pathId = 0;
/// <summary>
/// Gets the distributed file path.
/// </summary>
/// <param name="fileId">The file id.</param>
/// <returns></returns>

public static string GetDistributedFilePath(int fileId)
{
// normally you would store these in a config file or a database, however in keeping this example
// simple I have hard coded the network share paths
string[] networkSharePaths = new string[] {
@"C:\Temp\FileService\Share1\",
@"C:\Temp\FileService\Share2\",
@"C:\Temp\FileService\Share3\"}
;
// we use an internal variable to ensure there are no multithreading issues
int innerPathId = pathId;
// check that we have a valid path
pathId += 1;
if (pathId >= networkSharePaths.Length)
{
pathId 
= 0;
}

// check that we have a valid path for the internal variable, as we don't want
// to have a situation where innerPathId doesn't reference a valid network share paths
// array index
if (innerPathId >= networkSharePaths.Length)
{
innerPathId 
= 0;
}

// check the root path exists and creates the directory if it does not
if (!System.IO.Directory.Exists(networkSharePaths[innerPathId]))
{
System.IO.Directory.CreateDirectory(networkSharePaths[innerPathId]);
}

// pad the fileId to 9 places which allows for about 999,999,999 files
string paddedFileId = fileId.ToString().PadLeft(9'0');
// check main folder
string mainFolder = System.IO.Path.Combine(networkSharePaths[innerPathId], paddedFileId.Substring(03));
if (!System.IO.Directory.Exists(mainFolder))
{
System.IO.Directory.CreateDirectory(mainFolder);
}

// check the next folder down
string fileFolder = System.IO.Path.Combine(mainFolder, paddedFileId.Substring(33));
if (!System.IO.Directory.Exists(fileFolder))
{
System.IO.Directory.CreateDirectory(fileFolder);
}

// get the final file name (this will be in the xxx.extension format)
string finalFileName = paddedFileId.Substring(6);
// the result should be similar to the following
//  C:\Temp\FileService\Share1\
//      000
//          000
//              001.txt
//              002.txt
return System.IO.Path.Combine(fileFolder, finalFileName);
}

}

}

FileHandlerLinq.cs

The FileHandlerLinq class is a static class that each calling application can use to Insert/Select files from the Files table. For the purpose of this example update and deletes have been omitted.

Code

FileHandlerFactory.cs

The FileHandlerFactory is a simple factory class that determines if the browser should handle the file or if the file should be downloaded by the client as an attachment. In this example the factory is simple and should be customized for your own needs.

using System;
using System.Web;
namespace FileServiceCore
{
public static class FileHandlerFactory
{
/// <summary>
/// Transmits the file.
/// </summary>
/// <param name="context"><see cref="HttpContext"/>.</param>
/// <param name="fileId">The file id.</param>

public static void TransmitFile(HttpContext context, int fileId)
{
TransmitFile(context, fileId, 
false);
}

/// <summary>
/// Transmits the file.
/// </summary>
/// <param name="context"><see cref="HttpContext"/>.</param>
/// <param name="fileId">The file id.</param>
/// <param name="forceAsAttachment">if set to <c>true</c> [force as attachment].</param>

public static void TransmitFile(HttpContext context, int fileId, bool forceAsAttachment)
{
File file 
= FileHandlerLinq.Select(fileId);
if (file != null)
{
// parse the file name and get the extension, we use a ToUpperInvariant 
// as there are some characters that cannot be converted to lower case
string extension = file.NetworkPath.Substring(
file.NetworkPath.LastIndexOf(
".", StringComparison.OrdinalIgnoreCase
)
).ToUpperInvariant();
if (forceAsAttachment)
{
extension 
= string.Empty;
}

// in keeping this example simple we simply parse the file extension to determine if
// we should set the file as an attachment or send it back to the browser
switch (extension)
{
case ".JPG":
case ".JPEG":
case ".PNG":
case ".GIF":
case ".BMP":
TransmitFileToBrowser(context, file);
break;
default:
TransmitFileAsAttachment(context, file);
break;
}

}

}

/// <summary>
/// Transmits the file as attachment.
/// </summary>
/// <param name="context"><see cref="HttpContext"/>.</param>
/// <param name="file">The file entity class.</param>

private static void TransmitFileAsAttachment(HttpContext context, File file)
{
context.Response.AddHeader(
"content-disposition""attachment; filename=" + file.FileName);
context.Response.ContentType 
= file.ContentType;
context.Response.TransmitFile(file.NetworkPath);
}

/// <summary>
/// Transmits the file to browser.
/// </summary>
/// <param name="context"><see cref="HttpContext"/>.</param>
/// <param name="file">The file entity class.</param>

private static void TransmitFileToBrowser(HttpContext context, File file)
{
context.Response.ContentType 
= file.ContentType;
context.Response.TransmitFile(file.NetworkPath);
}

}

}

DownloadHandler.ashx and ImageHandler.ashx

The Generic ImageHander and DownloadHandlers are almost identical and their sole purpose is to parse the Uri for the FileId token. Once the FileId token is collected the generic handler calls for a file handler from the File Handler Factory.

The below sample .aspx page shows a common usage to add data to the file service.

Future Enhancements:

The following list highlights some of the possible enhancements that can be made to the service, depending on your needs:

  1. Public/Private flags for files
  2. Cache headers for the ImageHandler
  3. Resizing/cropping/effects for the ImageHandler
  4. Store statistics for file usage (last accessed, times accessed, etc)
  5. Demotion of non used files to another storage medium (tape, DVD, etc)
  6. File locking, to ensure files aren't accidentally deleted.

Assumptions:

  1. There is an available NAS/SAN with enough storage capacity for your needs
  2. The IIS account running the File Service has appropriate permissions to read/write to the NAS/SAN

本文转载 kane-nina
http://www.kanebarton.com/Samples/FileService/Default.aspx