You must Sign In to post a response.
  • Category: .NET

    Performance of handling zip file in memory using Zip Archive

    I am using Zip archive[System.IO.Compression] for collect the multiple xml memory stream and convert into zip memory stream then sent to remote server [where I am storing zip file].

    My intention is not to store any files[including xml file,zip file] in local machine,
    and not to use any 3rd party dll for compression.

    Below code is working fine and giving result as I expected. But still i am worrying about performance[In production env, we will get more records to zip storage]

    kindly advise the best solution in terms of performance and memory leak.

    What I have tried:
    Extension Method to convert List<any object> into XML

    public static MemoryStream Serialize<T>(this T dataToSerialize)
    {
    try
    {
    if (dataToSerialize == null) throw new ArgumentNullException();
    var serializer = new XmlSerializer(dataToSerialize.GetType());
    MemoryStream memstream = new MemoryStream();
    serializer.Serialize(memstream, dataToSerialize);

    return memstream;

    }
    }

    Extension method to convert into zip memory stream from List of different record.
    Input value like dir["xmlfilename",List<object data for xml conversion>]

    public static MemoryStream SerializeIntoZip<T>(this Dictionary<string,T> dataToSerialize)
    {
    var outStream = new MemoryStream();
    try
    {
    if (dataToSerialize == null) throw new ArgumentNullException();
    using (var archive = new ZipArchive(outStream, ZipArchiveMode.Create, true))
    {
    foreach(var data in dataToSerialize)
    {
    var fileInArchive = archive.CreateEntry(data.Key, CompressionLevel.Optimal);
    using (var entryStream = fileInArchive.Open())
    {

    using (var fileToCompressStream = data.Value.Serialize()) // Calling existing file stream method
    {
    entryStream.Flush();
    fileToCompressStream.Seek(0, SeekOrigin.Begin);
    fileToCompressStream.CopyTo(entryStream);
    fileToCompressStream.Flush();
    }
    }
    }
    }
    outStream.Seek(0, SeekOrigin.Begin);
    return outStream;
    }
    }

    My main method

    Dictionary<string, IList> listofobj =
    new Dictionary<string, IList>() {
    {"fileName_XX.xml", GetXXList1()},
    {"fileName_YY.xml", GetYYList()}};

    var mem = listofobj.SerializeIntoZip();

    //{
    // code to upload zip memory stream to S3 bucket
    //}
  • #769273
    Why can't you try FTP option. Like uploading the file to the server directly using FTP option so that you will not face any memory issue.

    something like below,


    using (WebClient FTPclient = new WebClient())
    {
    FTPclient.Credentials = new NetworkCredential(ftp_Username, ftp_Password);
    FTPclient.UploadFile("ftp://ftpserver/YourZip.zip", "STOR", localFilelocationPath);
    }


    Regards,
    Asheej T K

  • #769274
    Hi Asheej,

    Our requirement is very clear. we must use only Amazon S3 for storage [Also Amazon S3 code is ready and working as expected. So could not change anything ] . No FTP/SFTP..

    My questions is , best way to convert list of data into XML memory stream then need to zip it

    Please revert if require further clarification.


Sign In to post your comments