Skip to main content
added 76 characters in body
Source Link

I am looking for some efficient way to read from a large file and grep a string 'abracadabra' (=s1), here s1 is always present before eof(end of file) And if s1 is present search for another string 'P QRST'

i.e. Data of File f1:

ABCD EFGH IJKL MNOP
QRST UVW XY Z
...
...
abracadabra(eof)

I am novice for python, to do this I was using following method:

f = open('f1', 'r')
for line in f:
   if 'abracadabra' in line:
      print 'Found'
      found = True
      break
else:
   print 'EOF reached, string not found'        
f.close()

f = open('f1', 'r')
for line in f:
   if 'P QRST' in line:             # 'P QRST' can be present anywhere in the file(can be either in begging or just before s1)
      #do some operation

  

But this approach involves huge overhead for a large file data, as it read the files twice. Please suggest some efficient approach to do this.

Is there any efficient way like a 'grep' application in Linux shell?

I am looking for some efficient way to read from a large file and grep a string 'abracadabra' (=s1), here s1 is always present before eof(end of file) And if s1 is present search for another string 'P QRST'

i.e. Data of File f1:

ABCD EFGH IJKL MNOP
QRST UVW XY Z
...
...
abracadabra(eof)

I am novice for python, to do this I was using following method:

f = open('f1', 'r')
for line in f:
   if 'abracadabra' in line:
      print 'Found'
      found = True
      break
else:
   print 'EOF reached, string not found'        
f.close()

f = open('f1', 'r')
for line in f:
   if 'P QRST' in line:             # 'P QRST' can be present anywhere in the file(can be either in begging or just before s1)
      #do some operation

  

But this approach involves huge overhead for a large file data, as it read the files twice. Please suggest some efficient approach to do this.

I am looking for some efficient way to read from a large file and grep a string 'abracadabra' (=s1), here s1 is always present before eof(end of file) And if s1 is present search for another string 'P QRST'

i.e. Data of File f1:

ABCD EFGH IJKL MNOP
QRST UVW XY Z
...
...
abracadabra(eof)

I am novice for python, to do this I was using following method:

f = open('f1', 'r')
for line in f:
   if 'abracadabra' in line:
      print 'Found'
      found = True
      break
else:
   print 'EOF reached, string not found'        
f.close()

f = open('f1', 'r')
for line in f:
   if 'P QRST' in line:             # 'P QRST' can be present anywhere in the file(can be either in begging or just before s1)
      #do some operation

  

But this approach involves huge overhead for a large file data, as it read the files twice. Please suggest some efficient approach to do this.

Is there any efficient way like a 'grep' application in Linux shell?

deleted 38 characters in body
Source Link
Martijn Pieters
  • 1.1m
  • 325
  • 4.2k
  • 3.4k

I am looking for some efficient way to read from a large file and grep a string 'abracadabra''abracadabra' (=s1=s1), here s1s1 is always present before eof(end of file) And if s1s1 is present search for another string 'P QRST''P QRST'

i.e. Data of File f1: ABCD EFGH IJKL MNOP QRST UVW XY Z ... ... abracadabra(eof)

ABCD EFGH IJKL MNOP
QRST UVW XY Z
...
...
abracadabra(eof)

I am novice for python, to do this I was using following method:

1    f = open('f1', 'r')
2    for line in f:
3       if 'abracadabra' in line:
4          print 'Found'
5          found = True
6          break
7    else:
8       print 'EOF reached, string not found'        
9    f.close()
10    
11    f = open('f1', 'r')
12    for line in f:
13       if 'P QRST' in line:             # 'P QRST' can be present anywhere in the file(can be either in begging or just before s1)
14          #do some operation 

  

But this approach involves huge overhead for a large file data, as it read the files twice. Please suggest some efficient approach to do this.

I am looking for some efficient way to read from a large file and grep a string 'abracadabra'(=s1), here s1 is always present before eof(end of file) And if s1 is present search for another string 'P QRST'

i.e. Data of File f1: ABCD EFGH IJKL MNOP QRST UVW XY Z ... ... abracadabra(eof)

I am novice for python, to do this I was using following method:

1    f = open('f1', 'r')
2    for line in f:
3       if 'abracadabra' in line:
4          print 'Found'
5          found = True
6          break
7    else:
8       print 'EOF reached, string not found'        
9    f.close()
10    
11    f = open('f1', 'r')
12    for line in f:
13       if 'P QRST' in line:             # 'P QRST' can be present anywhere in the file(can be either in begging or just before s1)
14          #do some operation
  

But this approach involves huge overhead for a large file data, as it read the files twice. Please suggest some efficient approach to do this.

I am looking for some efficient way to read from a large file and grep a string 'abracadabra' (=s1), here s1 is always present before eof(end of file) And if s1 is present search for another string 'P QRST'

i.e. Data of File f1:

ABCD EFGH IJKL MNOP
QRST UVW XY Z
...
...
abracadabra(eof)

I am novice for python, to do this I was using following method:

f = open('f1', 'r')
for line in f:
   if 'abracadabra' in line:
      print 'Found'
      found = True
      break
else:
   print 'EOF reached, string not found'        
f.close()

f = open('f1', 'r')
for line in f:
   if 'P QRST' in line:             # 'P QRST' can be present anywhere in the file(can be either in begging or just before s1)
      #do some operation 

  

But this approach involves huge overhead for a large file data, as it read the files twice. Please suggest some efficient approach to do this.

Source Link

File handling in python for large amount of data

I am looking for some efficient way to read from a large file and grep a string 'abracadabra'(=s1), here s1 is always present before eof(end of file) And if s1 is present search for another string 'P QRST'

i.e. Data of File f1: ABCD EFGH IJKL MNOP QRST UVW XY Z ... ... abracadabra(eof)

I am novice for python, to do this I was using following method:

1    f = open('f1', 'r')
2    for line in f:
3       if 'abracadabra' in line:
4          print 'Found'
5          found = True
6          break
7    else:
8       print 'EOF reached, string not found'        
9    f.close()
10    
11    f = open('f1', 'r')
12    for line in f:
13       if 'P QRST' in line:             # 'P QRST' can be present anywhere in the file(can be either in begging or just before s1)
14          #do some operation
  

But this approach involves huge overhead for a large file data, as it read the files twice. Please suggest some efficient approach to do this.