# Near Real Time Mirroring v3

The RIPE NCC NRTM is a publicly available service that allows users to receive a stream of available data from the RIPE Database on a server. The user will receive a stream of data from the server with near real time updates. This service does not include any personal data.

# Pre-conditions:

# Steps

  • Stop whois using kill <PID>

      # NRTM server
  nrtm.enabled=false
  nrtm.update.interval=15
  # NRTM client
  nrtm.import.enabled=true
  nrtm.import.sources=RIPE-GRS
  nrtm.import.RIPE-GRS.source=RIPE
  nrtm.import.RIPE-GRS.host=whois.ripe.net
  nrtm.import.RIPE-GRS.port=4444

  • Update the last serial number in the mirrored database with RIPE.CURRENTSERIAL the using the following commands:

      mysql -udbint -p
  USE WHOIS_MIRROR_RIPE_GRS;
  SELECT max(serial_id) AS max_serial_id FROM serials;
  ## copy the result of the previous query and use it to the following query:
  UPDATE serials SET serial_id = RIPE.CURRENTSERIAL WHERE serial_id = max_serial_id;
  ALTER TABLE serials AUTO_INCREMENT = RIPE.CURRENTSERIAL;

  • Start whois using using the following command and check the output. Use -Ddump.total.size.limit to specify the dump size:

    /usr/bin/java -Dwhois -Djsse.enableSNIExtension=false -Dcom.sun.management.jmxremote -Dhazelcast.jmx=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=1099 -Xms1024m -Xmx8g -Dwhois.config=properties -Duser.timezone=UTC -Dhazelcast.config=hazelcast.xml -Dlog4j.configurationFile=file:log4j2.xml -jar whois.jar

you should see a log line like this:

2015-08-21 15:47:14,908 INFO  [NrtmClientFactory] Connected to whois.ripe.net:4444

if the above line is repeated multiple times, something is wrong. please stop the whois server.

  • Test that it worked by executing a query on the mirrored source using telnet:

      telnet localhost 1043
  -s RIPE-GRS 193.0.0.1 - 193.0.7.255

or by checking that the max(serial_id) in the database is bigger than RIPE.CURRENTSERIAL

mysql -udbint -p
select max(serial_id) from WHOIS_MIRROR_RIPE_GRS.serials

# NRTM and Keepalive

There is no support in the NRTM protocol for polling, so a client or server may not detect a socket disconnection for some time. Clients should set the TCP socket option SO_KEEPALIVE to enable the standard TCP keepalive mechanism. Clients should also set the keepalive interval option on every connection, as the system-wide default is very long. This option is platform-specific.

  • Linux

    • set SO_KEEPALIVE socket option.
    • System-wide default is 2 hours. Read the current value in seconds using: cat /proc/sys/net/ipv4/tcp_keepalive_time
    • The defaults can be changed on a per-socket basis
      • TCP_KEEPIDLE (the time to wait on an idle connection, before sending a keepalive probe).
      • TCP_KEEPINTVL (the time in seconds, between keepalive probes).
      • TCP_KEEPCNT (the maximum amount of probes to send, before dropping the connection).
    • Ref: http://www.manpages.info/linux/tcp.7.html
    • Ref: http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/overview.html

  • OSX

    • set SO_KEEPALIVE socket option.
    • System-wide default is 2 hours. Read the current value in ms using: /usr/sbin/sysctl net.inet.tcp.keepidle
    • The defaults can be changed on a per-socket basis
      • TCP_KEEPALIVE (the time to wait on an idle connection, before sending a keepalive probe).
    • Ref: https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man4/tcp.4.html

  • Windows

    • Set the SIO_KEEPALIVE_VALS control code.
    • Ref: https://msdn.microsoft.com/en-us/library/windows/desktop/ee470551(v=vs.85).aspx

# How to access the NRTM stream

You will find all technical requirements to access the stream and more information about NRTM in this document.

The RIPE NCC NRTM Service and its use is regulated by the RIPE Database Terms and Conditions and RIPE Database Acceptable Use Policy.

# Transferring To A Third Party

Transferring and/or making available of the NRTM Service and/or the data downloaded therefrom to a third party is only allowed, provided the following conditions are met:

  • The user obtains prior written permission from the RIPE NCC
  • The user ensures that third party accepts the RIPE Database Terms and Conditions for this purpose.
  • The user identifies the RIPE NCC as the source of the data and will explicitly state that this data is near-real time and may not reflect the latest data contained in the RIPE Database.
  • The user maintains the overall integrity of the data (e.g. will not misrepresent, falsify or exclude any data).

The RIPE NCC reserves the right to revoke this permission at any time if the user fails to meet any of these conditions.

# Contact

If you have any more questions, please contact us (opens new window).