# Near Real Time Mirroring v3

The RIPE NCC NRTM is a publicly available service that allows users to receive a stream of available data from the RIPE Database on a server. The user will receive a stream of data from the server with near real time updates. This service does not include any personal data.

# Pre-conditions:

# Steps

  • Stop whois using kill <PID>

      # NRTM server
      nrtm.enabled=false
      nrtm.update.interval=15
      # NRTM client
      nrtm.import.enabled=true
      nrtm.import.sources=RIPE-GRS
      nrtm.import.RIPE-GRS.source=RIPE
      nrtm.import.RIPE-GRS.host=whois.ripe.net
      nrtm.import.RIPE-GRS.port=4444
    
  • Update the last serial number in the mirrored database with RIPE.CURRENTSERIAL the using the following commands:

      mysql -udbint -p
      USE WHOIS_MIRROR_RIPE_GRS;
      SELECT max(serial_id) AS max_serial_id FROM serials;
      ## copy the result of the previous query and use it to the following query:
      UPDATE serials SET serial_id = RIPE.CURRENTSERIAL WHERE serial_id = max_serial_id;
      ALTER TABLE serials AUTO_INCREMENT = RIPE.CURRENTSERIAL;
    
  • Start whois using using the following command and check the output. Use -Ddump.total.size.limit to specify the dump size:

    /usr/bin/java -Dwhois -Djsse.enableSNIExtension=false -Dcom.sun.management.jmxremote -Dhazelcast.jmx=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=1099 -Xms1024m -Xmx8g -Dwhois.config=properties -Duser.timezone=UTC -Dhazelcast.config=hazelcast.xml -Dlog4j.configurationFile=file:log4j2.xml -jar whois.jar
    

you should see a log line like this:

2015-08-21 15:47:14,908 INFO  [NrtmClientFactory] Connected to whois.ripe.net:4444

if the above line is repeated multiple times, something is wrong. please stop the whois server.

  • Test that it worked by executing a query on the mirrored source using telnet:

      telnet localhost 1043
      -s RIPE-GRS 193.0.0.1 - 193.0.7.255
    

or by checking that the max(serial_id) in the database is bigger than RIPE.CURRENTSERIAL

mysql -udbint -p
select max(serial_id) from WHOIS_MIRROR_RIPE_GRS.serials

# NRTM and Keepalive

There is no support in the NRTM protocol for polling, so a client or server may not detect a socket disconnection for some time. Clients should set the TCP socket option SO_KEEPALIVE to enable the standard TCP keepalive mechanism. Clients should also set the keepalive interval option on every connection, as the system-wide default is very long. This option is platform-specific.

  • Linux

    • set SO_KEEPALIVE socket option.
    • System-wide default is 2 hours. Read the current value in seconds using: cat /proc/sys/net/ipv4/tcp_keepalive_time
    • The defaults can be changed on a per-socket basis
      • TCP_KEEPIDLE (the time to wait on an idle connection, before sending a keepalive probe).
      • TCP_KEEPINTVL (the time in seconds, between keepalive probes).
      • TCP_KEEPCNT (the maximum amount of probes to send, before dropping the connection).
    • Ref: http://www.manpages.info/linux/tcp.7.html
    • Ref: http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/overview.html
  • OSX

    • set SO_KEEPALIVE socket option.
    • System-wide default is 2 hours. Read the current value in ms using: /usr/sbin/sysctl net.inet.tcp.keepidle
    • The defaults can be changed on a per-socket basis
      • TCP_KEEPALIVE (the time to wait on an idle connection, before sending a keepalive probe).
    • Ref: https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man4/tcp.4.html
  • Windows

    • Set the SIO_KEEPALIVE_VALS control code.
    • Ref: https://msdn.microsoft.com/en-us/library/windows/desktop/ee470551(v=vs.85).aspx

# How to access the NRTM stream

You will find all technical requirements to access the stream and more information about NRTM in this document.

The RIPE NCC NRTM Service and its use is regulated by the RIPE Database Terms and Conditions and RIPE Database Acceptable Use Policy.

# Transferring To A Third Party

Transferring and/or making available of the NRTM Service and/or the data downloaded therefrom to a third party is only allowed, provided the following conditions are met:

  • The user obtains prior written permission from the RIPE NCC
  • The user ensures that third party accepts the RIPE Database Terms and Conditions for this purpose.
  • The user identifies the RIPE NCC as the source of the data and will explicitly state that this data is near-real time and may not reflect the latest data contained in the RIPE Database.
  • The user maintains the overall integrity of the data (e.g. will not misrepresent, falsify or exclude any data).

The RIPE NCC reserves the right to revoke this permission at any time if the user fails to meet any of these conditions.

# Contact

If you have any more questions, please contact us (opens new window).