Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce buffered I/O and replace getc with buffered read #647

Closed
wants to merge 2 commits into from

Conversation

pawosm-arm
Copy link
Collaborator

@pawosm-arm pawosm-arm commented Jan 3, 2019

This commit introduces 32k buffer for file I/O operations
and 32k buffer for fgetc() replacemnt routines.

This is follow-up of my work presented in pull request #642.

Some performance gain is observed and 'perf' trace does not
show getc operations at all, shifting the burden to actual
I/O system routines. In order to improve performance of these,
32k buffer is attached to every opened file by calling
setvbuf() function.

Due to suspicious way of passing pointers around in flang runtime,
all of the I/O buffers had to be allocated statically. In order to
prevent memory bloat, only limited number of simultaneously opened
files will be buffered, file descriptor ID is used for buffer
selection which is a thread-safe solution.

The test case I was using is following:

! *********************************************************
        program main

        implicit none
!       ---------------------------------------------------
        character(len=500) :: cart
        real(kind=8) :: t1,t2

!       ---------------------------------------------------
        open(unit=9,status='old',file='my_file.txt')
        open(unit=10,file='my_new_file.txt')
        call cpu_time(t1)
        do
!               read each line
                read(9,fmt='(A)') cart
!       ************************************************************
!       ************************************************************
!                       convert  process
!       ************************************************************
!       ************************************************************

                if(cart(1:4)=='/end') then
                        write(10,*) 'this is the end!'
                        exit
                else
                        write(10,*) cart
                endif
        enddo
        call cpu_time(t2)
        close(unit=9)
        close(unit=10)

        print*,' write and read :',t2-t1




!       ---------------------------------------------------

        end program main
! *********************************************************

my_file.txt.gz

Compiled with gfortran it gives much better timing results that when compiled with flang. My patch improves the timing of flang compiled program.

@pawosm-arm
Copy link
Collaborator Author

Anticipating your questions, setvbuf() is a very standard stdio.h function and you can find its documentation in Linux man pages and Visual Studio reference pages as well.

This commit introduces 32k buffer for file I/O operations
and 32k buffer for fgetc() replacemnt routines.

Some performance gain is observed and 'perf' trace does not
show getc operations at all, shifting the burden to actual
I/O system routines. In order to improve performance of these,
32k buffer is attached to every opened file by calling
setvbuf() function.

Due to suspicious way of passing pointers around in flang runtime,
all of the I/O buffers had to be allocated statically. In order to
prevent memory bloat, only limited number of simultaneously opened
files will be buffered, file descriptor ID is used for buffer
selection which is a thread-safe solution.

Signed-off-by: Paul Osmialowski <[email protected]>
… when really needed)

This is supplementary to the commit which introduces buffered I/O.

Signed-off-by: Paul Osmialowski <[email protected]>
@bryanpkc
Copy link
Collaborator

We agreed to close this one at the last call.

@bryanpkc bryanpkc closed this Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants