Create An Indexed Datetime From Date/time Info In 3 Columns Using Pandas

First off, here is a sample of my data, a csv with Year, Julian Day, 2400hr, and then 2 value columns. 2014,92,1931,6.234,10.14 2014,92,1932,5.823,9.49 2014,92,1933,5.33,7.65 2014,

Solution 1:

Try adding a parser to your read_csv

#assuming the order is year, month, day.  if you have time too, '%Y-%m-%d %H:%M:%S'    parser = lambda p: pd.datetime.strptime(p, '%Y-%m-%d')  

df = pd.read_csv('sorted.dat', 
                  parse_dates={'datetime': [1,2,3]}, 
                  date_parser=parser, 
                  header=None)

Update

Parser looks correct. I believe your current problem is in your read_csv(). The parse_dates arg is not formatted corrected (see excerpt from the doc string below).

If [1, 2, 3] -> try parsing columns 1, 2, 3 each as a separate date column.

If [[1, 3]] -> combine columns 1 and 3 and parse as a single date column.

So your parser was expecting all 3 columns at once, but was getting them 1 at a time. I discovered this when I added a print x statement to the parser func. Try this modification that uses a list of lists approach

data = pd.read_csv('sorted.dat',parse_dates=[[0,1,2]], date_parser=parser,index_col=0, header=None )