To be simple firmwares need reboots to be activated, drivers like nic drivers will give impact, teaming software if used will also create downtime. HBA drivers will disconnect your SAN drives so i would expect outage there also.
Agents and management should be rather painless as you don't interrupt production on your servers only possible monitoring. If you only update the firmware and "forget" the drivers you will also run into problems. Drivers can have a problem with underlying hardware if the firmware is updated. So my advice would be create maintenace windows and update the whole server if possible. If this is not an option don't change a working server....