博客
关于我
java 零拷贝 分块传输大文件(解除8M限制)
阅读量:264 次
发布时间:2019-03-01

本文共 3378 字,大约阅读时间需要 11 分钟。

NIO-Based Zero Copy File Transfer Implementation

Server Side Implementation

package com.atguigu.nio.zerocopy;import java.io.IOException;import java.net.InetSocketAddress;import java.net.ServerSocket;import java.nio.ByteBuffer;import java.nio.channels.ServerSocketChannel;import java.nio.channels.SocketChannel;public class NewIOServer {    public static void main(String[] args) throws Exception {        InetSocketAddress address = new InetSocketAddress(7002);        ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();        ServerSocket serverSocket = serverSocketChannel.socket();        serverSocket.bind(address);        ByteBuffer byteBuffer = ByteBuffer.allocate(4096);        while (true) {            SocketChannel socketChannel = serverSocketChannel.accept();            int readCount = 0;            while (-1 != readCount) {                try {                    readCount = socketChannel.read(byteBuffer);                } catch (Exception e) {                    break;                }                byteBuffer.rewind();            }        }    }}

Client Side Implementation

package com.atguigu.nio.zerocopy;import java.io.FileInputStream;import java.net.InetSocketAddress;import java.nio.channels.FileChannel;import java.nio.channels.SocketChannel;public class NewIOClient {    public static void main(String[] args) throws Exception {        SocketChannel socketChannel = SocketChannel.open();        socketChannel.connect(new InetSocketAddress("localhost", 7002));                String filename = "./test.avi";        FileChannel fileChannel = new FileInputStream(filename).getChannel();                long startTime = System.currentTimeMillis();        long totalSize = fileChannel.size();        long max_pkg = 8 * 1024 * 1024;        long cur_pos = 0;        long transSize = 0;                while (totalSize > max_pkg) {            long tmp = fileChannel.transferTo(cur_pos, max_pkg, socketChannel);            totalSize -= tmp;            cur_pos += tmp;            transSize += tmp;        }                if (totalSize > 0) {            long tmp = fileChannel.transferTo(cur_pos, totalSize, socketChannel);            transSize += tmp;        }                System.out.println("Total bytes transferred: " + transSize +                          " Elapsed time: " + (System.currentTimeMillis() - startTime));        fileChannel.close();    }}

Explanation

Server Side

  • Port Binding: The server binds to port 7002 using InetSocketAddress.
  • Reading Data: It uses a ByteBuffer of size 4096 to read data from connected clients.
  • Zero Copy Mechanism: The data is read directly from the socket channel into the buffer, leveraging zero copy functionality for efficient I/O operations.

Client Side

  • Connection Establishment: The client connects to the server at port 7002.
  • File Reading: It reads the file using FileInputStream and converts it to a FileChannel for NIO operations.
  • Zero Copy Transfer: The client transfers the file data to the server in chunks to handle large files efficiently. The transfer size is dynamically adjusted based on the maximum packet size (8MB in this implementation).
  • Performance Metrics: The total transfer time and bytes transferred are logged for verification purposes.

This implementation demonstrates the practical application of NIO's zero copy mechanism for high-performance file transfer scenarios.

转载地址:http://cqaa.baihongyu.com/

你可能感兴趣的文章
Python 使用哈希函数用于加密
查看>>
Python 依赖管理的革新——Poetry 深度解析
查看>>
python 保留精度及增加去除数字的千位分隔符(金额化数字)
查看>>
python 倒计时 9,8,7,。。。。。。0
查看>>
Python 入门开发学习笔记之数据的增删改查
查看>>
Python 入门教程(2)搭建环境 2.4、VSCode配置Node.js运行环境
查看>>
Python 八大排序算法合集
查看>>
python 关于epoll的学习
查看>>
Python 内存管理
查看>>
Python 内嵌函数:它们有什么用处?
查看>>
Python 内置 sum 函数 vs. for 循环性能
查看>>
Python 内置时间模块
查看>>
python 内部如何实现命名元组?
查看>>
python 写入json数据到数据库
查看>>
Python 列表删除相同的元素
查看>>
Python 初学者需要知道的四条建议
查看>>
Python 判断字符串是否包含子字符串
查看>>
python 判断当前时间是否为零点
查看>>
python 利用pandas读取本地中CSV文件的指定列 列名重命名 并保存回本地
查看>>
python 利用pyspark读取HDFS中CSV文件的指定列 列名重命名 并保存回HDFS
查看>>