newnix@exile.digital is a user on linuxrocks.online. You can follow them or interact with them if you have an account anywhere in the fediverse. If you don't, you can sign up here.
newnix@exile.digital @architect
Follow

I have no idea how I'm going to make nmap faster yet. It's taking about 3-5hrs to chew through the 3k+ hosts we need monitored now, which is nice enough, I guess, but it's still painfully slow atm.

Firing off multiple instances seemed to work, but I'm not sure if that negatively impacted accuracy. Even forcing batches of 128 hosts took at least 20min per batch to finish. Something's slowing down the processing, and I'm not entirely sure what. Maybe I need dtrace...

@architect something is probably not asynchronous in it and it's waiting for a timeout somewhere. Are you printing them first and moving on with no response?

@smallsees I'm running it with input from a file and logging to a file, so there shouldn't be any I/O bottlenecks unless ZFS is too slow on some SSDs, which seems unlikely. At this time, the most likely candidate seems to be `-Pn`, but I can't trust ping scans to validate targets, so I need to find out where things are slowing down and what can be done to work around it.

@architect O got it crawling /8 in a hour :) I scripted it to nmap -sP on a /24 grab the alive addresses then fork those into another process scan the ports I want and drop it to a html file while the original script starts looking for the next active addresses

@omnipotens That's pretty awesome, but I can't rely on pings to verify a host is alive, so I've been working with the ttl timing options to skip hosts if they don't respond within a reasonable window. Might be able to reduce that window some, but I feel like there's something else that I'm missing that can cut down on this latency.

@architect Well what really helps is forking the processes so your scanning multiple networks at once. and you might be able to use ping just not icmp but might be able to check response another way.

@architect I highly recommend writing a simple tcp connect script with a short timeout(<1s) and running that in parallel. using that method, I've checked ~4k connections in about 10-15 seconds.