pub struct Reader { /* private fields */ }
Expand description
Reader is designed to read data from given path in an asynchronous manner.
§Usage
Reader
provides multiple ways to read data from given reader.
Reader
implements Clone
so you can clone it and store in place where ever you want.
§Direct
Reader
provides public API including Reader::read
. You can use those APIs directly without extra copy.
use opendal::Operator;
use opendal::Result;
async fn test(op: Operator) -> Result<()> {
let r = op.reader("path/to/file").await?;
let bs = r.read(0..1024).await?;
Ok(())
}
§Read like Stream
use anyhow::Result;
use bytes::Bytes;
use futures::TryStreamExt;
use opendal::Operator;
async fn test(op: Operator) -> Result<()> {
let s = op
.reader("path/to/file")
.await?
.into_bytes_stream(1024..2048)
.await?;
let bs: Vec<Bytes> = s.try_collect().await?;
Ok(())
}
§Read like AsyncRead
and AsyncBufRead
use anyhow::Result;
use bytes::Bytes;
use futures::AsyncReadExt;
use opendal::Operator;
async fn test(op: Operator) -> Result<()> {
let mut r = op
.reader("path/to/file")
.await?
.into_futures_async_read(1024..2048)
.await?;
let mut bs = vec![];
let n = r.read_to_end(&mut bs).await?;
Ok(())
}
Implementations§
Source§impl Reader
impl Reader
Sourcepub async fn read(&self, range: impl RangeBounds<u64>) -> Result<Buffer>
pub async fn read(&self, range: impl RangeBounds<u64>) -> Result<Buffer>
Read give range from reader into Buffer
.
This operation is zero-copy, which means it keeps the bytes::Bytes
returned by underlying
storage services without any extra copy or intensive memory allocations.
Sourcepub async fn read_into(
&self,
buf: &mut impl BufMut,
range: impl RangeBounds<u64>,
) -> Result<usize>
pub async fn read_into( &self, buf: &mut impl BufMut, range: impl RangeBounds<u64>, ) -> Result<usize>
Sourcepub async fn fetch(&self, ranges: Vec<Range<u64>>) -> Result<Vec<Buffer>>
pub async fn fetch(&self, ranges: Vec<Range<u64>>) -> Result<Vec<Buffer>>
Fetch specific ranges from reader.
This operation try to merge given ranges into a list of
non-overlapping ranges. Users may also specify a gap
to merge
close ranges.
The returning Buffer
may share the same underlying memory without
any extra copy.
Sourcepub async fn into_stream(
self,
range: impl RangeBounds<u64>,
) -> Result<BufferStream>
pub async fn into_stream( self, range: impl RangeBounds<u64>, ) -> Result<BufferStream>
Create a buffer stream to read specific range from given reader.
§Notes
BufferStream is a zero-cost abstraction. It doesn’t involve extra copy of data.
It will return underlying Buffer
directly.
The Buffer
this stream yields can be seen as an iterator of [Bytes
].
§Inputs
range
: The range of data to read. range like..
it will read all data from reader.
§Examples
§Basic Usage
use std::io;
use bytes::Bytes;
use futures::TryStreamExt;
use opendal::Buffer;
use opendal::Operator;
use opendal::Result;
async fn test(op: Operator) -> io::Result<()> {
let mut s = op
.reader("hello.txt")
.await?
.into_stream(1024..2048)
.await?;
let bs: Vec<Buffer> = s.try_collect().await?;
// We can use those buffer as bytes if we want.
let bytes_vec: Vec<Bytes> = bs.clone().into_iter().flatten().collect();
// Or we can merge them into a single [`Buffer`] and later use it as [`bytes::Buf`].
let new_buffer: Buffer = bs.into_iter().flatten().collect::<Buffer>();
Ok(())
}
§Concurrent Read
The following example reads data in 256B chunks with 8 concurrent.
use std::io;
use bytes::Bytes;
use futures::TryStreamExt;
use opendal::Buffer;
use opendal::Operator;
use opendal::Result;
async fn test(op: Operator) -> io::Result<()> {
let s = op
.reader_with("hello.txt")
.concurrent(8)
.chunk(256)
.await?
.into_stream(1024..2048)
.await?;
// Every buffer except the last one in the stream will be 256B.
let bs: Vec<Buffer> = s.try_collect().await?;
// We can use those buffer as bytes if we want.
let bytes_vec: Vec<Bytes> = bs.clone().into_iter().flatten().collect();
// Or we can merge them into a single [`Buffer`] and later use it as [`bytes::Buf`].
let new_buffer: Buffer = bs.into_iter().flatten().collect::<Buffer>();
Ok(())
}
Sourcepub async fn into_futures_async_read(
self,
range: impl RangeBounds<u64>,
) -> Result<FuturesAsyncReader>
pub async fn into_futures_async_read( self, range: impl RangeBounds<u64>, ) -> Result<FuturesAsyncReader>
Convert reader into FuturesAsyncReader
which implements futures::AsyncRead
,
futures::AsyncSeek
and futures::AsyncBufRead
.
§Notes
FuturesAsyncReader is not a zero-cost abstraction. The underlying reader
returns an owned Buffer
, which involves an extra copy operation.
§Inputs
range
: The range of data to read. range like..
it will read all data from reader.
§Examples
§Basic Usage
use std::io;
use futures::io::AsyncReadExt;
use opendal::Operator;
use opendal::Result;
async fn test(op: Operator) -> io::Result<()> {
let mut r = op
.reader("hello.txt")
.await?
.into_futures_async_read(1024..2048)
.await?;
let mut bs = Vec::new();
r.read_to_end(&mut bs).await?;
Ok(())
}
§Concurrent Read
The following example reads data in 256B chunks with 8 concurrent.
use std::io;
use futures::io::AsyncReadExt;
use opendal::Operator;
use opendal::Result;
async fn test(op: Operator) -> io::Result<()> {
let mut r = op
.reader_with("hello.txt")
.concurrent(8)
.chunk(256)
.await?
.into_futures_async_read(1024..2048)
.await?;
let mut bs = Vec::new();
r.read_to_end(&mut bs).await?;
Ok(())
}
Sourcepub async fn into_bytes_stream(
self,
range: impl RangeBounds<u64>,
) -> Result<FuturesBytesStream>
pub async fn into_bytes_stream( self, range: impl RangeBounds<u64>, ) -> Result<FuturesBytesStream>
Convert reader into FuturesBytesStream
which implements futures::Stream
.
§Inputs
range
: The range of data to read. range like..
it will read all data from reader.
§Examples
§Basic Usage
use std::io;
use bytes::Bytes;
use futures::TryStreamExt;
use opendal::Operator;
use opendal::Result;
async fn test(op: Operator) -> io::Result<()> {
let mut s = op
.reader("hello.txt")
.await?
.into_bytes_stream(1024..2048)
.await?;
let bs: Vec<Bytes> = s.try_collect().await?;
Ok(())
}
§Concurrent Read
The following example reads data in 256B chunks with 8 concurrent.
use std::io;
use bytes::Bytes;
use futures::TryStreamExt;
use opendal::Operator;
use opendal::Result;
async fn test(op: Operator) -> io::Result<()> {
let mut s = op
.reader_with("hello.txt")
.concurrent(8)
.chunk(256)
.await?
.into_bytes_stream(1024..2048)
.await?;
let bs: Vec<Bytes> = s.try_collect().await?;
Ok(())
}