You must Sign In to post a response.
  • Category: Windows 8

    Read Data From Websites

    How To Read The Content Of Any Website That Provides Descriptive Information About What We had Search for And Then Store In Separate text File
  • #764702
    Hai Hardyk,
    Reading any data from the website needs crawling. You can create a small application in .Net windows and use the Web Browser control which will have the website you want to crawl.
    Once you do this setup, you need to get the element ids' from the webpage and then extract its values.
    Finally the extracted value can be saved in to the database or some file.
    Hope it will be helpful to you.

    Regards,
    Pawan Awasthi(DNS MVM)
    +91 8123489140 (whatsApp), +60 14365 1476(Malaysia)
    pawansoftit@gmail.com

  • #764720
    The technique which we are using for get the data from the website is called CRAWLING.

    You can see my following resource for more information.

    http://www.dotnetspider.com/resources/39155-Web-Crawler.aspx

    Following is the code.

    public static void Main(string[] args) {
    string Mylink = null;
    string Mystr;
    string Myanswer;
    int curPoint;
    if(args.Length != 1) {
    Console.WriteLine("Please use proper URL");
    return ;
    }
    string Myuristr = args[0];
    try {
    do {
    Console.WriteLine("Connecting to " + Myuristr);
    HttpWebRequest MyHttpWebRequest = (HttpWebRequest)
    WebRequest.Create(Myuristr);
    Myuristr = null;
    HttpWebResponse MyHttpWebResponse = (HttpWebResponse)
    MyHttpWebRequest.GetResponse();
    Stream MyInputString = MyHttpWebRequest.GetResponseStream();
    StreamReader MyStreamReader = new StreamReader(MyInputString);
    string Mystring = MyStreamReader.ReadToEnd();
    curPoint = 0;
    do {
    Mylink = FindMyLink(Mystring, ref curPoint);
    if(Mylink != null) {
    Console.WriteLine("Found the link : " + Mylink);
    Console.Write("Link, More, Quit?");
    answer = Console.ReadLine();

    if(string.Compare(answer, "L", true) == 0) {
    Myuristr = string.Copy(Mylink);
    break;
    } else if(string.Compare(answer, "Q", true) == 0) {
    break;
    } else if(string.Compare(answer, "M", true) == 0) {
    Console.WriteLine("Searching for another link.");
    }
    } else {
    Console.WriteLine("No link found.");
    break;
    }

    } while(Mylink.Length > 0);


    MyHttpWebResponse.Close();
    } while(Myuristr != null);

    }
    catch(Exception exc) {
    Console.WriteLine(exc.Message);
    }
    Console.WriteLine("Terminating Sample Crawler.");
    }

    static string FindMyLink(string MyHtmlstr,
    ref int MystartPoint) {
    int startPoint, endPoint;
    string Myuri = null;
    string Mylowcasestr = MyHtmlstr.ToLower();
    int i = Mylowcasestr.IndexOf("href=\"http", MystartPoint);
    if(i != -1) {
    startPoint = MyHtmlstr.IndexOf('"', i) + 1;
    endPoint = MyHtmlstr.IndexOf('"', startPoint);
    Myuri = MyHtmlstr.Substring(startPoint, endPoint-startPoint);
    MystartPoint = endPoint;
    }

    return Myuri;
    }
    }

    By Nathan
    Direction is important than speed

  • #764767
    Please go through below link and see if it helps.

    http://stackoverflow.com/questions/4758283/reading-data-from-a-website-using-c-sharp

    http://www.codeproject.com/Questions/176732/How-to-read-full-web-page-in-C


    Regards,
    Asheej T K
    Microsoft MVP[ASP.NET/IIS]
    DotNetSpider MVM


Sign In to post your comments