Hi, as promised at the last RIPE meeting, please find below a copy of the perl script I have which checks for the consistency between what is registered in the RIPE database for the domains and what is registered in the DNS. Do note, this script does nothing more than report what it thinks is inconsistencies, you still need to sift through the error reports this script produces and fix any inconsistencies yourselves. This script relies on a number of conventions and is a bit picky about what needs to be in the RIPE database if it is to not complain too much. First, the script has to know which domains it should check. It performs this decision by "traversing" the domain objects in the RIPE database starting from the object for the top-level domain. If the object in question has at least one sub-dom: attbibute, my script assumes that all names registered in this domain should be registered in the RIPE database with it's own domain object. This way, you use the "sub-dom" attbiutes to span out a subset of the DNS hierarchy under a given top-level domain where administrative registration in the RIPE database is required for DNS registration. The script subsequently does zone transfers for the DNS domains corresponding to the domains which should be checked as decided above. When querying the RIPE database, the set of name server records for each domain is recorded, so that the script can compare these with what is registered in the DNS. This is one point where my script deviates slightly from or is slightly more strict than the RIPE-49 document, in that it 1) does not permit IP addresses in the nserver: attribute (since you do not delegate to an IP address in the DNS, this makes sense) 2) does not permit more than one word in each nserver: attribute This makes registration such as: nserver: aun.uninett.no / 129.241.1.99 illegal, and nserver: aun.uninett.no nserver: 129.241.1.99 would also be flagged as an error when compared to the DNS output. It is my beleif that the RIPE database does not adequatly support the registration of glue records (I do not think it needs to either), and putting the glue record information in together in the nserver: attribute is bad when seen from the database normalization and maintenance side. If a domain is registerd in the DNS with only MX information, my script demands that the corresponding registration in the RIPE database should contain the string "MX-only" in one of the remarks: attributes of the domain. Subsequently, what has been harvested via the RIPE database and what has been fetched from the DNS is compared, and any inconsistencies are reported. These are: 1) Check for domains registered in the RIPE database but not in the DNS, report these as errors 2) Check for domains registered in the DNS but not in the RIPE database, report these as errors 3) For delegated domains, report any inconsistencies between the set of nserver: attribute values and the NS records in the DNS 4) Check for domains with both MX and NS records in the DNS, report these as errors 5) Check that only domains marked in the RIPE database as "MX-only" really have an MX-only registration in the DNS. Lastly, this script depends on dig version 2.0 (forget the one which comes with BIND 4.9.3 -- the external interface changed for the worse, IMHO) and on a whois client which uses getopt() to parse it's arguments (so that I can use the '--' construct to separate the options and the arguments). If you have another version of whois than this, the fix should be trivial (at the top of "get_ripe_data". Note: the script creates a "cache file" with name cc.ripe-cache in the current directory containing all the domain objects without sub-dom attributes, so that multiple runs in quick succession can be a little faster (do however remember to remove any objects you sent in updates for, or remove the cache after substantial modifications). Well, other than this, read the script -- it's some 450 lines of simple straight-forward Perl code. - Håvard #!/local/bin/perl # $Id: domain-check.pl,v 1.7 1995/03/12 18:37:22 he Exp $ # # Perl script to check the consistency between what is registered in the # RIPE database and what is registered in the DNS for the domain objects # under a given domain. Missing pieces (i.e. inconsistencies) for either # registration is reported. # Pull in data for a given domain from the DNS using dig sub get_dns_data { local($domain) = @_; local($ns, $label, $zoneok); local($wildcard_label); $zoneok = 0; $ns = &get_primary_ns($domain); if (!defined($ns)) { &err("Could not get SOA for $domain\n"); return undef; } $digcmd = "dig @$ns $domain axfr +nodefname +pfset=0xa024"; $digcmd = $digcmd . " | tr '[A-Z]' '[a-z]' |"; open(dig, $digcmd) || return undef; while(<dig>) { chop; split; if ( $_[1] eq "in" ) { chop($_[0]); $label = $_[0]; $wildcard_label = 0; if ($label =~ /^\*\./) { $label = substr($label, 2); $wildcard_label = 1; } } else { if (( $_[1] eq "matching" && $_[0] eq ";" ) || ( $_[1] eq "received" && $_[0] eq ";;" && $_[3] eq "records" )) { $zoneok = 1; } next; } if ( $_[2] eq "a" ) { $dns_glued{$label} = 1; } elsif ( $_[2] eq "ns" ) { chop($_[3]); $dns_ns{$label} = $dns_ns{$label} . ":" . $_[3]; $dns_delegated{$label} = 1; $in_dns{$label} = 1; } elsif ( $_[2] eq "mx" ) { $dns_mx_domain{$label} = 1; if ($wildcard_label) { $dns_mx_wildcard{$label} = 1; } $in_dns{$label} = 1; } elsif ( $_[2] eq "cname" ) { $dns_cname{$label} = 1; } elsif ( $_[2] eq "soa" ) { # nothing to do, ignore } elsif ( $_[2] eq "hinfo" ) { # nothing to do, ignore } elsif ( $_[2] eq "wks" ) { # nothing to do, ignore } else { printf stderr "spurious data: $_\n"; } } close(dig); if ($zoneok) { return 1; } else { &err("No matching SOA record found from $ns\n"); return undef; } } sub get_primary_ns { local($domain) = @_; local($ns, $digcmd); $digcmd = "dig $domain soa +nodefname +aa +pfset=0xa020"; $digcmd = $digcmd . " | tr '[A-Z]' '[a-z]' |"; open(dig, $digcmd) || return undef; while(<dig>) { chop; split; if ( $_[1] eq "soa" ) { close(dig); return $_[2]; } } close(dig); return undef; } # Pull in any cached data from the RIPE database previously saved to disk # by get_ripe_data sub get_ripe_cached_data { local($domain) = @_; local($tag); open(cache, $ripe_cache_name) || return undef; while (<cache>) { tr/A-Z/a-z/; chop; split; chop($_[0]); $tag = $_[0]; if ( $tag eq "nserver" ) { $ripe_ns{$domain} = $ripe_ns{$domain} . ":" . $_[1]; if ( $#_ > 1 ) { &err("RIPE: more than one nserver on a line for $domain\n"); } } if ( $tag eq "domain" ) { $domain = $_[1]; $in_ripe_db{$domain} = 1; } if ( $tag eq "sub-dom" ) { shift; while ($subdom = shift) { $fqdn = $subdom . "." . $domain; push(domains_to_check, $fqdn); if ( ! $has_subdom{$domain} ) { $has_subdom{$domain} = 1; push(needs_dns_xfer, $domain); } } } if ( $tag eq "remarks" ) { if ( /mx-only/ ) { $ripe_mx_only{$domain} = 1; } } } close(cache); return 1; } # Pull in data for a given domain from the RIPE database using whois # Put received data in "ripe-cache" (append it). The assumption is that # get_ripe_data will not be called unless data isn't found in the cache. sub get_ripe_data { local($domain) = @_; local($tag, $subdom, $fqdn); local($do_cache) = 1; local(@entry, $s); $whoiscmd = "whois -h whois.ripe.net -- '-r '$domain |"; open(whois, $whoiscmd) || return undef; open(cache, ">>$ripe_cache_name") || undef $do_cache; while(<whois>) { if ( /No entries found for the selected source/ ) { close(whois); return undef; } if ($do_cache) { push(@entry, $_); } tr/A-Z/a-z/; chop; split; chop($_[0]); $tag = $_[0]; if ( $tag eq "nserver" ) { $ripe_ns{$domain} = $ripe_ns{$domain} . ":" . $_[1]; if ( $#_ > 1 ) { &err("RIPE: more than one nserver on a line for $domain\n"); } } if ( $tag eq "domain" ) { $in_ripe_db{$domain} = 1; } if ( $tag eq "sub-dom" ) { if ($do_cache) { close(cache); } undef $do_cache; # Do not cache entries with sub-dom entries shift; while ($subdom = shift) { $fqdn = $subdom . "." . $domain; push(domains_to_check, $fqdn); if ( ! $has_subdom{$domain} ) { $has_subdom{$domain} = 1; push(needs_dns_xfer, $domain); } } } if ( $tag eq "remarks" ) { if ( /mx-only/ ) { $ripe_mx_only{$domain} = 1; } } } close(whois); if ($do_cache) { while($s = shift(@entry)) { print cache $s; } close(cache); } return 1; } # Check existence of data in RIPE database for a given domain # (independent of sub-dom tree). Might as well use code above # which pulls in data from RIPE database entry (if it exists). sub in_ripe_db { local($domain) = @_; return &get_ripe_data($domain); } # Get all RIPE data (decending recursively through sub-dom) # for a given domain, noting registered sub-doms that do not have # their own entry in the RIPE database sub get_all_ripe_data { local($domain) = @_; local(@domains_to_check) = @_; ¬ify("Getting cached RIPE data for domain $domain:"); $ripe_cache_name = $domain . "." . "ripe-cache"; &get_ripe_cached_data($domain); ¬ify("\nGetting all RIPE DB data: "); while ($domain = shift(domains_to_check)) { if (! $in_ripe_db{$domain} ) { # Not already seen in cache ¬ify_progress(); if ( ! &get_ripe_data($domain) ) { &err("RIPE database entry missing for $domain\n"); } } } } # Pull inn all relevant data for doing a check. Decend through sub-dom # entries in the RIPE database and pull in data for the relevant DNS zones # as well. sub pull_in_data { local($domain) = @_; &get_all_ripe_data($domain); ¬ify("Getting corresponding DNS data\n"); while ( $domain = shift(needs_dns_xfer) ) { ¬ify("Getting zone data for $domain\n"); if (! &get_dns_data($domain) ) { &err("Zone transfer of $domain failed\n"); } } } # Perform an extensive consistency check between the data fetched from DNS # and those fetched from the RIPE database. In particular, check: # # 1) Check that all entries in the RIPE database are in the DNS and # vice versa; report inconsistencies. # # 2) Where entries exist in both RIPE and the DNS, check: # # a) for delegated domains, the set of NS records in DNS and those # registered in the RIPE nserver entries match up # # b) for domains with both NS and MX records in the DNS, report # this as an inconsistency # # c) for MX-only domains (in the DNS), check that the corresponding # entry in the RIPE database contains the remark "MX-only". sub do_consistency_check { &report_missing_ripe(); &report_missing_dns(); &report_ns_inconsistencies(); &report_inconsistent_domains(); &report_mx_comments(); } # Report entries in the DNS which do not exist in the RIPE database sub report_missing_ripe { local($domain); foreach $domain (keys %in_dns) { if ( ! $in_ripe_db{$domain} ) { if ( &in_ripe_db($domain) ) { &err("Missing as sub-dom in parent object: $domain\n"); } else { &err("Missing in RIPE db: $domain\n"); } $ignore{$domain} = 1; } } } # Report entries in the RIPE database missing in the DNS sub report_missing_dns { local($domain); foreach $domain (keys %in_ripe_db) { if ( ! $in_dns{$domain} ) { if ( ! $dns_cname{$domain} ) { &err("Missing in DNS: $domain\n"); $ignore{$domain} = 1; } } } } sub split_ns_list { local($ns_str) = @_; local(%array, @tmp, $elt); $ns_str =~ s/^://; @tmp = split(/[:]+/, $ns_str); while ($elt = shift(@tmp)) { $array{$elt} = 1; } return %array; } # Report inconsistencies between registered name servers DNS/RIPE sub report_ns_inconsistencies { local(%dns_nses, %ripe_nses); local($heading_written) = 0; local($domain, $ns); foreach $domain (keys %in_dns) { if ( ! $ignore{$domain} ) { %dns_nses = &split_ns_list($dns_ns{$domain}); %ripe_nses = &split_ns_list($ripe_ns{$domain}); foreach $ns (keys %dns_nses) { if ( ! $ripe_nses{$ns} ) { &err("Missing NS in RIPE for $domain: $ns\n"); } } foreach $ns (keys %ripe_nses) { if ( ! $dns_nses{$ns} ) { &err("Missing NS in DNS for $domain: $ns\n"); } } } } } # Report domains with both NS and MX records in the DNS # and domains in RIPE database with both nserver and remark: MX-only. sub report_inconsistent_domains { local($domain); foreach $domain (keys %in_dns) { if (defined($dns_ns{$domain}) && defined($dns_mx_domain{$domain})) { &err("Both MX and NS records for $domain in DNS\n"); } } foreach $domain (keys %in_ripe_db) { if (defined($ripe_mx_only{$domain}) && defined($ripe_ns{$domain})) { &err("Both nserver and remark: MX-only in RIPE db for $domain\n"); } } } # Report MX-only domains where RIPE entry doesn't contain remark "MX-only" # and MX-only domains in RIPE database which are fully delegated in DNS sub report_mx_comments { local($domain); foreach $domain (keys %in_dns) { if ($in_ripe_db{$domain}) { if ($dns_mx_domain{$domain}) { if (! $ripe_mx_only{$domain}) { &err("MX-domain in DNS, RIPE db not marked for $domain\n"); } } } } foreach $domain (keys %in_dns) { if ($dns_delegated{$domain}) { if ($ripe_mx_only{$domain}) { &err("MX-domain in RIPE, DNS fully delegated for $domain\n"); } } } } sub err { local(@args) = @_; if ($dot_written) { $dot_written = 0; printf stderr "\n"; } printf stderr @args; } sub notify { local(@args) = @_; if ($dot_written) { $dot_written = 0; printf stderr "\n"; } printf stderr @args; } sub notify_progress { printf stderr "."; $dot_written = 1; } $arg = $ARGV[0]; if (defined($arg) && ($arg =~ /^..$/)) { &pull_in_data($arg); &do_consistency_check(); } else { &err("usage: domain-check.pl cc\n"); &err("\twhere cc is a two-letter country code\n"); }