Bagaimana cara mengatasi terlalu banyak pengecualian file terbuka di flink saat menggunakan konektor pencarian elastis untuk mengirim data aliran ke indeks pencarian elastis?

Saya menggunakan konektor pencarian elastis flink untuk mengirim data ke indeks pencarian elastis.

Tapi saya tidak dapat menjalankan pekerjaan dan mendapatkan pengecualian ini. Bagaimana cara mengatasi masalah ini?. Apakah ada pengaturan atau konfigurasi yang saya lewatkan? Adakah yang bisa membimbing saya?

io.netty.channel.ChannelException: The operating system does not offer enough file handles to open the network connection. Please increase the number of of available file handles.
    at org.apache.flink.runtime.io.network.netty.NettyClient.connect(NettyClient.java:207)
    at org.apache.flink.runtime.io.network.netty.PartitionRequestClientFactory.createPartitionRequestClient(PartitionRequestClientFactory.java:81)
    at org.apache.flink.runtime.io.network.netty.NettyConnectionManager.createPartitionRequestClient(NettyConnectionManager.java:60)
    at org.apache.flink.runtime.io.network.partition.consumer.RemoteInputChannel.requestSubpartition(RemoteInputChannel.java:112)
    at org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.requestPartitions(SingleInputGate.java:419)
    at org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.getNextBufferOrEvent(SingleInputGate.java:441)
    at org.apache.flink.streaming.runtime.io.BarrierTracker.getNextNonBlocked(BarrierTracker.java:85)
    at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:192)
    at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:63)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:261)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:665)
    at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.ChannelException: Failed to open a socket.
    at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:62)
    at io.netty.channel.socket.nio.NioSocketChannel.<init>(NioSocketChannel.java:72)
    at sun.reflect.GeneratedConstructorAccessor8.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.Class.newInstance(Class.java:442)
    at io.netty.bootstrap.AbstractBootstrap$BootstrapChannelFactory.newChannel(AbstractBootstrap.java:453)
    at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:306)
    at io.netty.bootstrap.Bootstrap.doConnect(Bootstrap.java:133)
    at io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:115)
    at org.apache.flink.runtime.io.network.netty.NettyClient.connect(NettyClient.java:198)
    ... 11 more
Caused by: java.net.SocketException: Too many open files
    at sun.nio.ch.Net.socket0(Native Method)
    at sun.nio.ch.Net.socket(Net.java:411)
    at sun.nio.ch.Net.socket(Net.java:404)
    at sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:105)
    at sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:60)
    at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:60)
    ... 21 more

person Dhinesh    schedule 08.06.2017    source sumber


Jawaban (1)


Untuk menambah jumlah pegangan file di sistem Anda, Anda dapat mengaturnya melalui ulimit -n untuk suatu proses jika Anda memiliki hak akses root. Jika Anda ingin mengubah jumlah pegangan file di seluruh sistem, ubah nilainya di /proc/sys/fs/file-max (di Linux).

person Till Rohrmann    schedule 08.06.2017