The Wayback Machine - https://web.archive.org/web/20200830030234/https://github.com/vaexio/vaex/issues/912
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE-REQUEST] Support netCDF-4 files #912

Open
snitramodranoel opened this issue Jul 22, 2020 · 2 comments
Open

[FEATURE-REQUEST] Support netCDF-4 files #912

snitramodranoel opened this issue Jul 22, 2020 · 2 comments

Comments

@snitramodranoel
Copy link

@snitramodranoel snitramodranoel commented Jul 22, 2020

Any suggestions on how to make vaex support nc-4 files, which are themselves backed by hdf5?

When attempting to read a netCDF-4 file with a given variable in (x,y,z,t) coordinates this is what I get:

ValueError: array is of length 360, while the length of the DataFrame is 2928

This netCDF-4 file contains 360 latitude values, and 2928 time points, as evidenced by ncdump -h:

netcdf GSWP3.BC.Rainf.3hrMap.1940 {
dimensions:
	time = UNLIMITED ; // (2928 currently)
	lat = 360 ;
	lon = 720 ;
variables:
	double time(time) ;
		time:long_name = "Time" ;
		time:standard_name = "time" ;
		time:calendar = "proleptic_gregorian" ;
		time:units = "hours since 1871-01-01 00:00:00" ;
	float lat(lat) ;
		lat:long_name = "Latitude" ;
		lat:standard_name = "latitude" ;
		lat:axis = "Y" ;
		lat:units = "degrees_north" ;
	float lon(lon) ;
		lon:long_name = "Longitude" ;
		lon:standard_name = "longitude" ;
		lon:axis = "X" ;
		lon:units = "degrees_east" ;
	float Rainf(time, lat, lon) ;
		Rainf:_fillvalue = 1.e+20 ;
		Rainf:long_name = "Rainfall rate" ;
		Rainf:standard_name = "rainfall_flux" ;
		Rainf:alma_name = "Rainf" ;
		Rainf:amip_name = "prra" ;
		Rainf:units = "kg m-2 s-1" ;
// global attributes:
		:title = "Global Soil Wetness Project 3 <http://hydro.iis.u-tokyo.ac.jp/GSWP3> EXP1 forcing data" ;
		:id = "GSWP3.BC.Rainf.3hrMap.1940" ;
		:creator = "Hyungjun Kim <hjkim@iis.u-tokyo.ac.jp>" ;
		:institution = "Institute of Industrial Science, The University of Tokyo" ;
		:contributors = "Satoshi Watanabe, Eun-Chul Chang, Nobuyuki Utsumi, Kei Yoshimura, Gilbert Compo, Hirabayashi Yukiko, James Famiglietti, and Taikan Oki" ;
		:conventions = "ALMA-3, CF-1.6, AMIP" ;
		:history = "| 0.5b | 2014-06-22 | first beta release\n| 0.6b | 2014-07-18 | remove negative values from Qair\n| 0.7b | 2014-10-15 | add netCDF description\n|                   | beta version release\n| 0.8b | 2017-05-12 | correct wind using CRU-CL2.0\n| 0.9b | 2017-05-13 | correct bias of reference Prcp (GPCC v7) in early 20th Century\n|                   | reproduce Rainf/Snowf corresponding to updated wind speed\n| 1.0  | 2017-06-01 | version 1 official release\n| 1.05 | 2018-02-17 | timespan extended (1901-2014)\n|                   | fill wind values over ocean pixels with downscaled reanalysis\n| 1.06 | 2018-07-06 | change variable naming convention: amip_name [wss] -> [sfcWind]" ;
}
@norlandrhagen
Copy link

@norlandrhagen norlandrhagen commented Jul 23, 2020

Seconded! This support would be really useful for weather/climate/oceanography data.

@maartenbreddels
Copy link
Member

@maartenbreddels maartenbreddels commented Aug 18, 2020

Yeah, should be possible. Once we have #865 in, it should also be easier to add new file formats. Would be a good example/usecase on how to add that. Anyone willing to cooperate/work on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.