Questions tagged [csv]
Files containing data arrange in a table, often with commas (hence Comma Separated Values), to separate columns. Rows are separated by newlines (but not all newlines are row separators as fields can be quoted to contain the separator newlines. Use this tag for full-fledged CSV data not the simpler case of one record per line or completely unquoted (use csv-simple for that kind of data).
948 questions
1
vote
4
answers
132
views
try to split csv with multiple headers
I have a csv from a solar inverter and I need to input the data into a SQL database.
The problem I have is that two inverters are in the same csv, so I need to split the csv into two or "do ...
- 21
-3
votes
2
answers
160
views
Add a character to duplicate emails using bash only
Input data:
id,location_id,name,title,[email protected],department
1,1,Susan houston,Director of Services,[email protected],
2,1,Christina Gonzalez,Director,[email protected],
3,2,Brenda brown,"...
-1
votes
6
answers
186
views
How to count blank fields from a delimited file in Unix
from script below:
EmpID:Name:Designation:UnitName:Location:DateofJoining:Salary
1001:Thomson:SE:IVS:Mumbai:10-Feb-1999:60000
1002:Johnson:TE::Bangalore:18-Jun-2000:50000
1003:Jackson:DM:IMS:Hyderabad:...
- 183
9
votes
5
answers
1k
views
Run command on each line of CSV file, using fields in different places of the command
I have a CSV file and want to run a command for each line, using the fields of the file as separate arguments.
For example given the following file:
foo,42,red
bar,13,blue
baz,27,green
I want to run ...
- 312
5
votes
5
answers
583
views
Unix shell scripting help- how to replace file delimiter if exist inside double quotes data of a delimited csv file?
My input file data:
"Per Sara Porras.|, LLC"|column2_data|column3_data
column1_data|"column2|data"|"column3|data"
Required output:
"Per Sara Porras.@@@, LLC"|...
-4
votes
5
answers
193
views
Command to display all the employees whose first name have more than 6 characters
From the script below I need to know the following:
EmpNo#Email#Name#JobLevel#Experience
641357#Amrit_Mohanty#Amrit Mohanty#3#2
678522#Puneet_Mishra#Puneet Mishra#3#1
670242#Vikas_Bharti#Vikas Bharti#...
- 183
3
votes
2
answers
681
views
How to extract lines by condition from large CSV?
I have a CSV file with 100 million rows, approximately 60GB in size.
Now, I want to create another CSV from it but only take lines that match a certain condition: at the end of the CSV, there is an ...
- 490
0
votes
4
answers
304
views
AWK command not working as expected giving & data as output
I have file where I need to find if some column is empty. File is delimited by ;
Some of data contains & in word and awk is command it not working for same .
Lets say I have file (...
- 101
3
votes
3
answers
367
views
Large file manipulation
I have a Garmin Nuvi which uses OpenStreet maps. Garmin do postcodes, but is usually 2-3 years out of date for Scotland. OSM does not do British postcodes, but the Post office does, and it can be ...
- 49
-3
votes
2
answers
115
views
Using awk, append a value in last field in csv file based upon value in specific field
input.csv -
"family_guy","Brian","b47f0a80-d848-4d81-a45b-7ba930e6048b","son"
"family_guy","Brian","b47f0a80-d848-4d81-a45b-...
- 553
0
votes
5
answers
1k
views
command-line tool to sum the values in a column of a CSV file
I am looking for a command-line tool to calculate the sum of the values in a specified column of a CSV file. (Update: The CSV file might have quoted fields, so a simple solution just to break on a ...
- 649
1
vote
1
answer
118
views
Print csv columns according to user input
I have a .csv file with multiple lines with the following format:
¬Country¬,¬Year¬,¬Singer¬,¬Song Title¬
¬Japan¬,¬1999¬,¬Utada Hikaru¬,¬First Love¬
¬South Korea¬,¬1999¬,¬Lee Jung Hyun¬,¬Wa¬
...
I can ...
- 27
3
votes
5
answers
251
views
How to remove double quotes within the double-quoted field values in .dat file
I have a text file that has around 15 columns.
The fields are separated by comma.
One column that is description is double-quoted
and also has some words which are double-quoted.
I need to retain ...
- 43
2
votes
7
answers
3k
views
How to remove embedded newlines from CSV fields
I am struggling with processing a file. Normally I should have a line for each host definition. But sometimes, someone split some fields on different lines.
Here is an example:
"host1","...
1
vote
1
answer
110
views
Add list of urls to the Qutebrowser’s sqlite3 history
General overview
I have a `history.csv CSV file of urls formated like this:
<url>;<title>;2024-03-30T12:00:00;2024-03-30T12:00:00
(The 2024-03-30T12:00:00 is the same)
The goal is to get ...
- 1,529