File reading and writing

Hi,
I am trying to read and write to file which is very large (writing 500 MB of data into file after computation). MY program is taking a lot of time more than 35 minutes. Is it possible to reduce the running time? ur help will be appreciated

I don't know. I haven't seen your code.

Post it, and we'll have a look.

(use [highlight] tags)

hey its not possible to post the code ..
What i tried to do is do some computation then write into a file in a for loop for 70,000 entries or so. Its takin a lot of time I cant figure out for if writin is takin much time or the computation. Computation in my context is not CPU intensive so I am guessin writin is takin long time

When you get into big files little programming things that typically don't matter in terms of processing time (most of my programs take milliseconds) matter. What language are you using? There's probably some places you could streamline things to get it to run faster, but without seeing anything there's not much help we can provide. Is it writing each time it runs through the loop? you could make it go much faster by storing in a temporary variable and then write every x loops (to prevent accidental data loss/speed things up) or at the end.

If you want help with your program we really need to see some code. You don't have to post the entire application, just the relevant snippet and you can obfuscate it if you need to.

File I/O is slow. Find a better way to do it, such as batching writes (writing a lot of data at once). Avoid opening and closing the stream after each write, avoid having to seek write position. Write in a binary format...

These are probably the easiest ways to optimize writes to file...

Also, if you're pulling the information from a database, indexes will greatly improve performance on a SELECT query.