博客
关于我
java 零拷贝 分块传输大文件(解除8M限制)
阅读量:264 次
发布时间:2019-03-01

本文共 3378 字,大约阅读时间需要 11 分钟。

NIO-Based Zero Copy File Transfer Implementation

Server Side Implementation

package com.atguigu.nio.zerocopy;import java.io.IOException;import java.net.InetSocketAddress;import java.net.ServerSocket;import java.nio.ByteBuffer;import java.nio.channels.ServerSocketChannel;import java.nio.channels.SocketChannel;public class NewIOServer {    public static void main(String[] args) throws Exception {        InetSocketAddress address = new InetSocketAddress(7002);        ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();        ServerSocket serverSocket = serverSocketChannel.socket();        serverSocket.bind(address);        ByteBuffer byteBuffer = ByteBuffer.allocate(4096);        while (true) {            SocketChannel socketChannel = serverSocketChannel.accept();            int readCount = 0;            while (-1 != readCount) {                try {                    readCount = socketChannel.read(byteBuffer);                } catch (Exception e) {                    break;                }                byteBuffer.rewind();            }        }    }}

Client Side Implementation

package com.atguigu.nio.zerocopy;import java.io.FileInputStream;import java.net.InetSocketAddress;import java.nio.channels.FileChannel;import java.nio.channels.SocketChannel;public class NewIOClient {    public static void main(String[] args) throws Exception {        SocketChannel socketChannel = SocketChannel.open();        socketChannel.connect(new InetSocketAddress("localhost", 7002));                String filename = "./test.avi";        FileChannel fileChannel = new FileInputStream(filename).getChannel();                long startTime = System.currentTimeMillis();        long totalSize = fileChannel.size();        long max_pkg = 8 * 1024 * 1024;        long cur_pos = 0;        long transSize = 0;                while (totalSize > max_pkg) {            long tmp = fileChannel.transferTo(cur_pos, max_pkg, socketChannel);            totalSize -= tmp;            cur_pos += tmp;            transSize += tmp;        }                if (totalSize > 0) {            long tmp = fileChannel.transferTo(cur_pos, totalSize, socketChannel);            transSize += tmp;        }                System.out.println("Total bytes transferred: " + transSize +                          " Elapsed time: " + (System.currentTimeMillis() - startTime));        fileChannel.close();    }}

Explanation

Server Side

  • Port Binding: The server binds to port 7002 using InetSocketAddress.
  • Reading Data: It uses a ByteBuffer of size 4096 to read data from connected clients.
  • Zero Copy Mechanism: The data is read directly from the socket channel into the buffer, leveraging zero copy functionality for efficient I/O operations.

Client Side

  • Connection Establishment: The client connects to the server at port 7002.
  • File Reading: It reads the file using FileInputStream and converts it to a FileChannel for NIO operations.
  • Zero Copy Transfer: The client transfers the file data to the server in chunks to handle large files efficiently. The transfer size is dynamically adjusted based on the maximum packet size (8MB in this implementation).
  • Performance Metrics: The total transfer time and bytes transferred are logged for verification purposes.

This implementation demonstrates the practical application of NIO's zero copy mechanism for high-performance file transfer scenarios.

转载地址:http://cqaa.baihongyu.com/

你可能感兴趣的文章
PySpark - 在数据框中求和一列并将结果返回为 int
查看>>
pyspark On Yarn 的模块依赖问题
查看>>
PySpeech(Python)-转录MP3文件?
查看>>
Pyspider WebUI 未授权访问致远程代码执行漏洞复现
查看>>
pyspider爬虫学习-文档翻译-Architecture.md
查看>>
pytesseract.pytesseract.TesseractNotFoundError: tesseract is not installed or it‘s not in your path
查看>>
Pytesseract集字符白名单
查看>>
PyTorch DataLoader 报错 Segmentation fault (core dumped) 的原因及解决方案
查看>>
Pytest ------ 标记函数
查看>>
Pytest Fixture的妙用:深度解析范围和实际运用!
查看>>
Pytest monkeypatch不适用于导入的函数
查看>>
Pytest 命令行报错: Hint: make sure your test modules/packages have valid Python names.
查看>>
pytest 的 request fixture:实现个性化测试需求
查看>>
Pytest+Allure 接口自动化测试平台开发实战
查看>>
pytest+allure使用动态级别,参数化severity
查看>>
Pytest+Unittest+Git+Jenkins企业级CICD自动化测试平台建设方案
查看>>
Pytest+Yaml 数据驱动测试用例
查看>>
pytest-base-url插件之配置可选的项目系统URL
查看>>
pytest-xdist 进行多进程并发测试!
查看>>
pytest-xdist:远程多主机 - 分布式运行自动化测试
查看>>