Sri Lanka Equity Forum
Dear Reader,

Registration with the Sri Lanka Equity Forum would enable you to enjoy an array of other services such as Member Rankings, User Groups, Own Posts & Profile, Exclusive Research, Live Chat Box etc..

All information contained in this forum is subject to Disclaimer Notice published.


Thank You
Sri Lanka Equity Forum

Discussion Forum for Stock Market Investors in Sri Lanka

Sri Lanka News
Forum Disclaimer

The information contained in this forum have been submitted by third parties directly without any verification by us. The information available in this forum is not researched or purported to be complete description of the subject matter referred to herein. We do not under any circumstances whatsoever guarantee the accuracy and completeness information contained herein.

www.srilankaequity.com its blogs, forums, subdomains and/or its affiliates and/or its web masters, administrators or moderators shall not in any way be responsible or liable for loss or damage which any person or party may sustain or incur by relying on the contents of this report and acting directly or indirectly in any manner whatsoever.

Trading or investing in stocks & commodities is a high risk activity. Any action you choose to take in the markets is totally your own responsibility, www.srilankaequity.com its blogs, forums, subdomains and/or its affiliates and/or its web masters, administrators or moderators shall not be liable for any, direct or indirect, consequential or incidental damages or loss arising out of the use of this information. The information on this website is neither an offer to sell nor solicitation to buy any of the securities mentioned herein. The writers may or may not be trading in the securities mentioned.

You are not connected. Please login or register

Sri Lanka Equity Forum » Stock Market & Forum Help » CSE EOD Data !

CSE EOD Data !

Go to page : Previous  1, 2

View previous topic View next topic Go down  Message [Page 2 of 2]

1 CSE EOD Data ! on Sun Jun 05, 2011 11:37 pm

CubedSol

avatar
Manager - Equity Analytics
Manager - Equity Analytics
First topic message reminder :

Hi Guys ,

I'm back to the forums scene , Sorry was inactive for a considerable period of time . Nice to see the NEW home has grown to be THE home for all/Majority of the guys who were at other sites.

As some of you guys know I was using AmiBroker and EOD data from the investnow.lk for analysis...But now that the site is down , and yours truly NOT being active there I don't have a way to get the EOD data .

I do have an updated EXCEL/CSV file containing EOD data until 2011/01/11 and I can export the current days data through DFN , But I need to fill the gap .

SO , If anyone can answer the question "How to export the HISTORY data from DFN ?? "" Will be saving my life...

Also pls find the mentioned data file for your reference.

http://www.4shared.com/file/JQ7W8v3v/All_Stocks_Colombo_Stock_Excha.html


Thanks & Best Regards,
CubedSol



Last edited by CubedSol on Mon Jun 06, 2011 10:38 pm; edited 1 time in total (Reason for editing : EOD Data file re-uploaded)


51 Re: CSE EOD Data ! on Thu Aug 04, 2011 8:48 am

JennyFunny


Senior Manager - Equity Analytics
Senior Manager - Equity Analytics
Guys i managed to sought the problem out..the errors we due to incorrect date format.
This is 4th August 2011,should be inserted as 20110804 (yyyymmdd)

52 Re: CSE EOD Data ! on Thu Aug 04, 2011 1:44 pm

gayanr600


Equity Analytic
Equity Analytic
CubedSol wrote:Updated files containing data until 03/08/2011 are located below .

Database Backup :

RAR : 2.8MB
http://www.4shared.com/file/wlNnyPsl/MyNewData__Updated_03_08_2011_.html

ZIP : 3.2MB
http://www.4shared.com/file/tEh3Eb9m/MyNewData__Updated_03_08_2011_.html

PS - ASPI & MPI updated until 02/08/2011

It is much grateful, if you can add the "CSE DL.xlsm" file too.

53 Re: CSE EOD Data ! on Thu Aug 04, 2011 11:52 pm

CubedSol


Manager - Equity Analytics
Manager - Equity Analytics
Dear Gayan ,

No worries I've uploaded the file and you may download the same from the below link .

http://www.4shared.com/file/n_xtQ_DW/CSE_DL.html

PS - IMHO , After going through all the above posts from all our fellow members it seems that the reason for all these errors is because EVERYONE is is using their own DB structure... (Eg - DATE format) . So I thout that it'd be better for everyone IF I upload a MIRROR COPY of my DB daily , So that yall can SIMPLY replace your exsisting DB folder with .

54 Re: CSE EOD Data ! on Fri Aug 05, 2011 12:08 am

CubedSol


Manager - Equity Analytics
Manager - Equity Analytics
Razz thanks for the -REPS given to me.... Razz

55 Re: CSE EOD Data ! on Fri Aug 05, 2011 12:54 am

xhora


Senior Manager - Equity Analytics
Senior Manager - Equity Analytics
CubedSol wrote:Razz thanks for the -REPS given to me.... Razz

hey keep uploading mate
+ Rep from me

56 Re: CSE EOD Data ! on Fri Aug 05, 2011 8:29 am

smallville

avatar
Associate Director - Equity Analytics
Associate Director - Equity Analytics
SanjivFund wrote:Guys i managed to sought the problem out..the errors we due to incorrect date format.
This is 4th August 2011,should be inserted as 20110804 (yyyymmdd)

I've been telling this for many times.. why nobody check it first? Twisted Evil Date format plays a huge role in plotting correct chart movements in Amibroker.. U mess it up u get all ur charts wrong..

Its very easy for me to download the daily file from CSE site and keep uploading it with this 1 minute workout of arranging the fields in Symbol-Date-Open-Hi-low-close-vol and deleting the unnecessary fields.. Its a manual workout daily..Or at least I make sure I download the file and keep it even if I dont update it daily..

Its a great pleasure if someone with programming knowledge with PHP/JS/etc.. can write a script (VB or whatever) to grab the csv file from CSE site and mail it to our mailboxes..

57 Re: CSE EOD Data ! on Fri Aug 05, 2011 8:42 am

gayanr600

avatar
Equity Analytic
Equity Analytic
Thanx man,
You r doing a great job.
This is my +rep from me.

58 Re: CSE EOD Data ! on Fri Aug 05, 2011 9:32 am

JennyFunny


Senior Manager - Equity Analytics
Senior Manager - Equity Analytics
ha ha cubes..great uploads thanks for taking the time
+rep from me

59 Re: CSE EOD Data ! on Fri Aug 05, 2011 9:33 am

JennyFunny


Senior Manager - Equity Analytics
Senior Manager - Equity Analytics
@smallville great idea..can anyone do it?

60 Re: CSE EOD Data ! on Sat Aug 06, 2011 11:34 am

CubedSol

avatar
Manager - Equity Analytics
Manager - Equity Analytics
Updated files containing data until 05/08/2011 are located below .

Database Backup :

RAR : 2.8MB
http://www.4shared.com/file/U_FuFRaT/MyNewData__Updated_05_08_2011_.html

ZIP : 3.2MB
http://www.4shared.com/file/7VRwBobC/MyNewData__Updated_05_08_2011_.html

CSE DL.xlsm
http://www.4shared.com/file/n_xtQ_DW/CSE_DL.html

PS - ASPI & MPI updated until 05/08/2011

61 Re: CSE EOD Data ! on Sat Aug 06, 2011 7:54 pm

ssuni

avatar
Manager - Equity Analytics
Manager - Equity Analytics
CubedSol wrote:

CSE DL.xlsm
http://www.4shared.com/file/n_xtQ_DW/CSE_DL.html

PS - ASPI & MPI updated until 05/08/2011
thanks cube

62 Re: CSE EOD Data ! on Mon Aug 08, 2011 11:16 pm

CubedSol

avatar
Manager - Equity Analytics
Manager - Equity Analytics
Updated files containing data until 08/08/2011 are located below .

Database Backup :

RAR : 2.8MB
http://www.4shared.com/file/145vE99m/MyNewData__Updated_08_08_2011_.html

ZIP : 3.2MB
http://www.4shared.com/file/d6lN-rYu/MyNewData__Updated_08_08_2011_.html

CSE DL.xlsm
http://www.4shared.com/file/n_xtQ_DW/CSE_DL.html

PS - ASPI & MPI updated until 05/08/2011

63 Re: CSE EOD Data ! on Tue Aug 09, 2011 9:13 am

webhoundx


Stock Analytic
Stock Analytic
Thanks for the upload CubedSol

smallville wrote:
Its a great pleasure if someone with programming knowledge with PHP/JS/etc.. can write a script (VB or whatever) to grab the csv file from CSE site and mail it to our mailboxes..

Currently working on this. I have a made a google spreadsheet that will auto update between 3pm-4pm every weekday with CSE site's CSV data. I was planning to test the process this week and let you guys know... Emailing part I'm currently working on. (A bit hard to find time with work and all Sad )

64 Re: CSE EOD Data ! on Tue Aug 09, 2011 9:20 am

JennyFunny


Senior Manager - Equity Analytics
Senior Manager - Equity Analytics
thanks..
+rep

65 Re: CSE EOD Data ! on Tue Aug 09, 2011 8:22 pm

twimalasuriya

avatar
Senior Equity Analytic
Senior Equity Analytic
webhoundx wrote:Thanks for the upload CubedSol

smallville wrote:
Its a great pleasure if someone with programming knowledge with PHP/JS/etc.. can write a script (VB or whatever) to grab the csv file from CSE site and mail it to our mailboxes..

Currently working on this. I have a made a google spreadsheet that will auto update between 3pm-4pm every weekday with CSE site's CSV data. I was planning to test the process this week and let you guys know... Emailing part I'm currently working on. (A bit hard to find time with work and all Sad )

Guys.

I have already developed a similar script that run as a cron job ( a scheduler running on linux ) each day (excluding holidays) , download the EOD, data, convert to excel , and then mail it to csereports@googlegroups.com, so the members of the google group get a copy.

I have another script that hourly runs at 10am till 3pm that gives same data, for the owner of http://samueldaniel7.blogspot.com/

If anyone wants I can easily develop a script to send EOD data in metastock format itself.

If anyone is interested I can share my scripts too. Its written in php.

66 Re: CSE EOD Data ! on Tue Aug 09, 2011 8:37 pm

twimalasuriya

avatar
Senior Equity Analytic
Senior Equity Analytic
[quote="smallville"]
SanjivFund wrote:
Its a great pleasure if someone with programming knowledge with PHP/JS/etc.. can write a script (VB or whatever) to grab the csv file from CSE site and mail it to our mailboxes..

@smallville - if you send me a mail to twimalasuriya at gmail dot com, I can add you to the maling list of http://groups.google.com/group/csereports where I already send such a auto generated EOD excel file.

67 Re: CSE EOD Data ! on Tue Aug 09, 2011 9:29 pm

webhoundx


Stock Analytic
Stock Analytic
twimalasuriya wrote:
Guys.

I have already developed a similar script that run as a cron job ( a scheduler running on linux ) each day (excluding holidays) , download the EOD, data, convert to excel , and then mail it to csereports@googlegroups.com, so the members of the google group get a copy.

I have another script that hourly runs at 10am till 3pm that gives same data, for the owner of http://samueldaniel7.blogspot.com/

If anyone wants I can easily develop a script to send EOD data in metastock format itself.

If anyone is interested I can share my scripts too. Its written in php.

I was searching for a script but couldn't find anything developed for CSE. That's why I came up with a system of my own.
Anyways great news. Really appreciate you willing to share the sources as well Smile
Now if only we can come up with a system to auto import the data to Ami as well... cheers

68 Re: CSE EOD Data ! on Tue Aug 09, 2011 10:04 pm

twimalasuriya

avatar
Senior Equity Analytic
Senior Equity Analytic
webhoundx wrote:
I was searching for a script but couldn't find anything developed for CSE. That's why I came up with a system of my own.
Anyways great news. Really appreciate you willing to share the sources as well Smile
Now if only we can come up with a system to auto import the data to Ami as well... cheers


I have below files on my host server (3ix), and I run downloadSummary.php as a cron job at 3pm (SL time) to mail this file to me.

By the way, I'm not a php developer but used php for this because its the best suited for the job.

downloadSummary.php

Code:


<?php

require_once ("support_functions.php");

$url = 'http://www.cse.lk/listedcompanies/overview.htm?d-16544-e=2&6578706f7274=1';
$dir = '/home/wimalasu/';
$file_name = 'summary_'.date("Y-m-d").'.tsv' ;
$excel_file_name = 'summary_'.date("Y-m-d").'.xls' ;

if (isholiday() == 0) {

// download tsv
downloadFile($url , $dir , $file_name);


// convert to excel

convertToExcel($dir.$file_name ,$dir.$excel_file_name );

// send it by email.

sendemail($file = $dir.$excel_file_name , 'thiwankaw@gmail.com' , 'cse_summary' , 'application/excel');


}else{

echo 'today is a cse holiday';

}



?>


support_functions.php

Code:


<?php


function sendemail($file,$to,$subject,$contenttype){

// Require Pear Mail Packages
require_once ("/home/wimalasu/php/Mail.php");
require_once ("/home/wimalasu/php/Mail/mime.php");
$recipients  = $to;
// Additional headers
$headers["From"] = 'thiwankaw@gmail.com';
$headers["To"]    = $to;
$headers["Subject"] = $subject;
$crlf = "\n";

$mime = new Mail_mime($crlf);
$mime->addAttachment($file, $contenttype);

$message = $mime->get();
$headers = $mime->headers($headers);
$params["host"]    = 'ssl://smtp.gmail.com';
$params["auth"]    = TRUE; // note: there are *no delimiters*
$params["port"] = 465 ;

// note: there are *no delimiters* for DIGEST-MD5 either.
// If you want to use PLAIN,
// you have to use delimiters like this: 'PLAIN'
// $params["auth"]    = DIGEST-MD5;
$params["username"]    = 'thiwankaw@gmail.com';
$params["password"]    = 'mypassword here because i have to send authenticated email';
//$params["localhost"]= '127.0.0.1';
// Debug so that we see what's happenning for the moment.
$params["debug"]    = "True";
// create the mail object using the Mail::factory method
$mail_message =& Mail::factory('smtp', $params);
$mail_message->send ($recipients, $headers, $message);
}


// this downloads a file to local folder
function downloadFile($url , $dir , $file_name){

    if($file_name == NULL){ $file_name = basename($url);}
    $url_stuff = parse_url($url);
    $port = isset($url_stuff['port']) ? $url_stuff['port'] : 80;

    $fp = fsockopen($url_stuff['host'], $port);
    if(!$fp){ return false;}

    $query  = 'GET ' . $url_stuff['path'] .'?' . $url_stuff['query'] . " HTTP/1.0\n";

    $query .= 'Host: ' . $url_stuff['host'];
    $query .= "\n\n";

    fwrite($fp, $query);

    while ($tmp = fread($fp, 8192))  {
        $buffer .= $tmp;
    }

    preg_match('/Content-Length: ([0-9]+)/', $buffer, $parts);
    $file_binary = substr($buffer, - $parts[1]);
    if($file_name == NULL){
        $temp = explode(".",$url);
        $file_name = $temp[count($temp)-1];
    }
    $file_open = fopen($dir . "/" . $file_name,'w');
    if(!$file_open){ return false;}
    fwrite($file_open,$file_binary);
    fclose($file_open);

}

// check if a given day is a holiday for cse
function isholiday(){

$day_index = date("w");
// it's the weekend
if($day_index == 0 || $day_index == 6) {
  return 1;
}


$handle = @fopen("/home/wimalasu/holidays", "r");
$isholiday = 0;
$today = date("Y-m-d");
if ($handle) {
    while (($buffer = fgets($handle, 4096)) !== false) {

      if ( strcmp($today,trim($buffer)) == 0 ){
          $isholiday = 1;
      }
    }
    if (!feof($handle)) {
        echo "Error: unexpected fgets() fail\n";
    }
    fclose($handle);
}
  return $isholiday;
}


function convertToExcel($inputFileName,$outputFileName) {

require_once 'phpexcel/Classes/PHPExcel.php';


$inputFileType = 'CSV';

/**  Create a new Reader of the type defined in $inputFileType  **/
$objReader = PHPExcel_IOFactory::createReader($inputFileType);
/**  Set the delimiter to a TAB character  **/
$objReader->setDelimiter("\t");
/**  Load the file to a PHPExcel Object  **/
$objPHPExcel = $objReader->load($inputFileName);

/** Freeze pane so that the heading line won't scroll **/
$objPHPExcel->getActiveSheet()->freezePane('A2');



$writer = new PHPExcel_Writer_Excel5($objPHPExcel);
$writer->save($outputFileName);


}




?>



holidays (a list of holidays, so we dont download EOD file for these days)
Code:


2011-01-19
2011-02-04
2011-02-16
2011-02-17
2011-03-02
2011-04-13
2011-04-14
2011-04-15
2011-05-02
2011-05-17
2011-05-18
2011-06-15
2011-07-14
2011-08-31
2011-10-11
2011-10-26
2011-11-10
2011-12-10
2011-12-26




69 Re: CSE EOD Data ! on Wed Aug 10, 2011 9:28 am

JennyFunny


Senior Manager - Equity Analytics
Senior Manager - Equity Analytics
Great stuff seriously..thanks for the information..

70 Re: CSE EOD Data ! on Thu Sep 01, 2011 5:00 pm

Genting


Senior Manager - Equity Analytics
Senior Manager - Equity Analytics
can someone pls share latest EOD data for Amibroker

71 Re: CSE EOD Data ! on Sun Sep 04, 2011 7:40 pm

krishaja


Stock Trader
@twi malasooriy
pl.. tell me where to add these codes?

72 Re: CSE EOD Data ! on Sun Sep 04, 2011 7:51 pm

smallville

avatar
Associate Director - Equity Analytics
Associate Director - Equity Analytics
twimalasuriya wrote:
webhoundx wrote:
I was searching for a script but couldn't find anything developed for CSE. That's why I came up with a system of my own.
Anyways great news. Really appreciate you willing to share the sources as well Smile
Now if only we can come up with a system to auto import the data to Ami as well... cheers


I have below files on my host server (3ix), and I run downloadSummary.php as a cron job at 3pm (SL time) to mail this file to me.

By the way, I'm not a php developer but used php for this because its the best suited for the job.

downloadSummary.php



Greek to me Wink pardon my ignorance since I hate programming.. he hee.. but great job.. Send u a mail.. pls do the needful.. thanks..

73 Re: CSE EOD Data ! on Mon Sep 05, 2011 10:47 am

twimalasuriya

avatar
Senior Equity Analytic
Senior Equity Analytic
krishaja wrote:@twi malasooriy
pl.. tell me where to add these codes?


krishaja , you have two options

1. run these scripts on your server as a scheduled job
2. get a copy of the email i'm generating each day at https://groups.google.com/forum/?hl=en#!forum/csereports

if you want to go for option 1, you might need some php programming knowledge. if you have, and just want some help , then PM me.

if you are happy with option 2, goto https://groups.google.com/forum/?hl=en#!forum/csereports and become a member.

(If you are privacy conscious [understandably], then i suggest you create a new gmail address just for this and add it to the mail group. then just add a filter the the new email address to forward the email to your regular email )

please do not hesitate to PM me if you have any questions.

74 Re: CSE EOD Data ! on Tue Sep 06, 2011 8:58 am

ha_na


Manager - Equity Analytics
Manager - Equity Analytics
I need 9 February 2011 and 9 may 2011 EOD data for all companies. I have error on those two days. Can someone help me?
Thanks

75 Re: CSE EOD Data ! on Tue Sep 06, 2011 9:29 am

twimalasuriya

avatar
Senior Equity Analytic
Senior Equity Analytic
ha_na wrote:I need 9 February 2011 and 9 may 2011 EOD data for all companies. I have error on those two days. Can someone help me?
Thanks

I just copied two files you wanted to the location below

(Symbol Date (YYMMDD) Open High Low Close Vol )

http://wimalasuriya.com/cse/eod/20110209.txt
http://wimalasuriya.com/cse/eod/20110509.txt

These are from a local database I update each day from data taken from cse.lk

I run a sql like below to get the file

Code:


SELECT  share,DATE_FORMAT(date,'%Y%m%d') , open,high,low,close,sharevolume
INTO OUTFILE '/tmp/20110509.txt'
  FIELDS TERMINATED BY '\t'
  LINES TERMINATED BY '\n'
  from ohlc  where date = '2011-5-9' ;



Last edited by twimalasuriya on Tue Sep 06, 2011 9:36 am; edited 1 time in total (Reason for editing : added file format)

76 Re: CSE EOD Data ! on Tue Sep 06, 2011 10:49 am

ha_na


Manager - Equity Analytics
Manager - Equity Analytics
thanks my dear.
I want excel format as I want to convert it to Metastock. Is it possible?
Thanks.

twimalasuriya wrote:
ha_na wrote:I need 9 February 2011 and 9 may 2011 EOD data for all companies. I have error on those two days. Can someone help me?
Thanks

I just copied two files you wanted to the location below

(Symbol Date (YYMMDD) Open High Low Close Vol )

http://wimalasuriya.com/cse/eod/20110209.txt
http://wimalasuriya.com/cse/eod/20110509.txt

These are from a local database I update each day from data taken from cse.lk

I run a sql like below to get the file

Code:


SELECT  share,DATE_FORMAT(date,'%Y%m%d') , open,high,low,close,sharevolume
INTO OUTFILE '/tmp/20110509.txt'
  FIELDS TERMINATED BY '\t'
  LINES TERMINATED BY '\n'
  from ohlc  where date = '2011-5-9' ;

77 Re: CSE EOD Data ! on Tue Sep 06, 2011 12:13 pm

twimalasuriya

avatar
Senior Equity Analytic
Senior Equity Analytic
ha_na wrote:thanks my dear.
I want excel format as I want to convert it to Metastock. Is it possible?
Thanks.


pm me a email address, i will send you

78 Re: CSE EOD Data ! on Mon Sep 26, 2011 4:57 pm

seyon


Assistant Vice President - Equity Analytics
Assistant Vice President - Equity Analytics
Thanks

79 Re: CSE EOD Data ! on Sun Oct 16, 2011 7:56 pm

amilaela

avatar
Senior Manager - Equity Analytics
Senior Manager - Equity Analytics
okk

http://www.facebook.com/profile.php?id=100001851586843

80 Re: CSE EOD Data ! on Sat Nov 26, 2011 1:11 am

aero


Manager - Equity Analytics
Manager - Equity Analytics
can someone pls share latest ASPI & MPI EOD data for Amibroker?

81 Re: CSE EOD Data ! on Sat Nov 26, 2011 1:56 am

bakapandithaya

avatar
Vice President - Equity Analytics
Vice President - Equity Analytics
SanjivFund wrote:thanks..
+rep
Can we give + rep again?

View previous topic View next topic Back to top  Message [Page 2 of 2]

Go to page : Previous  1, 2

Permissions in this forum:
You cannot reply to topics in this forum