How to Configure OSPF Authentication on Cisco (MD5, Step by Step)

Step 1 — Understand what MD5 authentication actually protects

OSPF "MD5" authentication does not encrypt your routing updates — anyone capturing the link still sees the LSAs in clear. What it does is prove authenticity: each packet includes a message digest (a hash) computed over the packet plus a shared secret key. A neighbor recomputes the digest with its own copy of the key; if the two match, the packet is trusted, and if they do not, the packet is silently discarded.

The practical effect: a rogue or misconfigured router that does not know the key can send Hellos all day and never form an adjacency, because your legitimate routers drop its packets before any neighbor relationship starts. That is exactly the outcome you want — segments stay closed to route injection without you having to track MAC addresses or ports.

Note the terminology. In classic IOS the command literally says md5, and the key-id lets you rotate keys without dropping the adjacency (during a rollover a router accepts packets signed with any configured key). On newer IOS-XE the same protection is available with stronger HMAC-SHA algorithms via key chains, covered in Step 5.

Step 2 — Configure interface-level MD5 on both ends

Interface-level MD5 is the approach most CCNA/CCNP material teaches because it is explicit: you turn authentication on per interface, and you can see exactly which links are protected. Two commands are required on each interface facing a neighbor. The first defines the key: ip ospf message-digest-key 1 md5 <key>, where 1 is the key-id and <key> is the shared secret (up to 16 characters for MD5, and case-sensitive). The second tells the interface to actually use message-digest authentication.

Do this on R1's interface, then apply the identical key-id and key on R2's matching interface. The key-id (1) and the key string must be byte-for-byte identical on both ends. If you only configure the key but forget ip ospf authentication message-digest, the interface still uses whatever the area default is (typically none), so both commands matter.

Because you are changing authentication on a live segment, expect the adjacency to drop and re-form as the second side is completed. That is normal — it will settle once both ends match.

! ----- R1 -----
R1(config)# interface GigabitEthernet0/0
R1(config-if)# ip ospf message-digest-key 1 md5 GoldfishOSPF
R1(config-if)# ip ospf authentication message-digest
R1(config-if)# end

! ----- R2 (must match key-id, type, and key) -----
R2(config)# interface GigabitEthernet0/0
R2(config-if)# ip ospf message-digest-key 1 md5 GoldfishOSPF
R2(config-if)# ip ospf authentication message-digest
R2(config-if)# end

Step 3 — Alternative: enable MD5 for the whole area

If you want every interface in an area authenticated without visiting each one, enable it area-wide under the OSPF process with area <id> authentication message-digest. This sets the default authentication type to message-digest for all OSPF interfaces in that area, so you no longer type ip ospf authentication message-digest on each interface — but you still must define the key per interface with ip ospf message-digest-key, because the key itself is never global.

Configure the area command on every router that participates in the area (each router only knows about the areas it is in), and put the matching key on each interface in that area. Miss the area command on one router and its interfaces default back to no authentication, so its neighbors will discard its packets.

The two methods interoperate through a clear precedence rule: an interface-level ip ospf authentication ... command overrides the area default on that interface. Use that to your advantage — for example, if an area is message-digest by default but one interface must stay unauthenticated (say, to a device that cannot do MD5), set ip ospf authentication null on just that interface.

! ----- R1: make area 0 message-digest by default -----
R1(config)# router ospf 1
R1(config-router)# area 0 authentication message-digest
R1(config-router)# exit

! Still define the key on each interface in the area:
R1(config)# interface GigabitEthernet0/0
R1(config-if)# ip ospf message-digest-key 1 md5 GoldfishOSPF
R1(config-if)# end

! Optional: exempt one interface from the area default
R1(config)# interface GigabitEthernet0/1
R1(config-if)# ip ospf authentication null

Step 4 — Verify authentication and the adjacency

Confirm two things: that the interface is enforcing message-digest authentication, and that the neighbor reached FULL. Start with show ip ospf interface <int>. In the output, look for the line Message digest authentication enabled, followed by Youngest key id is 1 — that confirms both the type and the key-id are active on this interface. If it instead says No authentication or Simple password authentication enabled, your Step 2/Step 3 configuration did not take on that interface.

Then run show ip ospf neighbor. A healthy authenticated adjacency looks exactly like an unauthenticated one — State reaches FULL. Authentication is invisible when it is working; you only notice it when a key is wrong and the neighbor never appears. Verifying from both routers is worth the extra command, because a one-sided misconfiguration can leave one end thinking all is well while the other drops every packet.

If the neighbor is missing or hung, jump to the Common problems section — the fix is almost always a key, key-id, or type mismatch.

R1# show ip ospf interface GigabitEthernet0/0
GigabitEthernet0/0 is up, line protocol is up
  Internet Address 10.0.12.1/24, Area 0, Attached via Network Statement
  ...
  Message digest authentication enabled
    Youngest key id is 1

R1# show ip ospf neighbor
Neighbor ID     Pri   State      Dead Time   Address      Interface
2.2.2.2           1   FULL/DR    00:00:36    10.0.12.2    GigabitEthernet0/0

Step 5 — Newer option: SHA authentication with a key chain (IOS-XE)

MD5 is fine for CCNA/CCNP exam scope and still widely deployed, but it is a legacy hash. On modern IOS-XE you can authenticate OSPFv2 with HMAC-SHA algorithms by pointing the interface at a key chain instead of a single md5 key. Build a key chain, give it a key number, set the key-string (the secret) and the cryptographic-algorithm (for example hmac-sha-256), then bind it to the interface with ip ospf authentication key-chain <name>.

As always, both ends must match — same key number, same key-string, and the same algorithm — and both routers must run an IOS-XE image that supports the OSPF key-chain feature. Availability and the exact set of hmac-sha-* algorithms vary by platform and release, so confirm against Cisco's current documentation for your specific IOS-XE version rather than assuming a given box supports it.

Practicing this method needs real IOS-XE, not a simulator. Packet Tracer emulates command behavior and does not implement the full key-chain SHA feature; run it on physical gear or in Cisco Modeling Labs (CML), which is a virtual appliance you deploy in a supported hypervisor such as VMware — not a desktop program. If you use CML at home, check Cisco's current site for licensing, node limits, and the exact IOS-XE images it ships.

! ----- R1 (repeat identically on R2) -----
R1(config)# key chain OSPF-SHA
R1(config-keychain)# key 1
R1(config-keychain-key)# key-string GoldfishSHA256
R1(config-keychain-key)# cryptographic-algorithm hmac-sha-256
R1(config-keychain-key)# exit
R1(config-keychain)# exit
R1(config)# interface GigabitEthernet0/0
R1(config-if)# ip ospf authentication key-chain OSPF-SHA
R1(config-if)# end

! Verify as in Step 4:
R1# show ip ospf interface GigabitEthernet0/0
R1# show ip ospf neighbor

Common problems (and the fix)

Key or key-id mismatch. This is the number-one cause. If the key strings differ (remember they are case-sensitive) or the key-ids differ, every OSPF packet from the other side is discarded. Most often the neighbor never appears at all in show ip ospf neighbor, because Hellos are dropped before any relationship begins; in some scenarios you may instead catch it hung in EXSTART/EXCHANGE and never reaching FULL. Fix: make the key-id, the type, and the key identical on both ends. debug ip ospf adj prints an explicit "Mismatched Authentication Key" message that names the offending key-id.

Authentication enabled on only one end. If R1 requires message-digest but R2 sends unauthenticated packets (or vice versa), the requiring side drops the other's Hellos and the adjacency never forms. Fix: enable the same authentication type on both ends of every link, and verify from both routers with show ip ospf interface.

Type mismatch (null vs plain-text vs message-digest). ip ospf authentication (plain-text), ip ospf authentication message-digest (MD5), and ip ospf authentication null are three different types. Two neighbors set to different types will not authenticate even if a key is present. Fix: agree on one type per link; for MD5 both sides need ip ospf authentication message-digest.

Inconsistent area-wide vs interface settings. Turning on area 0 authentication message-digest but forgetting the per-interface key — or forgetting the area command on one router while others have it — leaves some interfaces defaulting to no authentication. Fix: pick one method per area and apply it everywhere; remember the interface-level command overrides the area default, and the key must always be set per interface. show ip ospf interface tells you the type actually in effect on each link.

Key configured but authentication never turned on. Setting ip ospf message-digest-key without ip ospf authentication message-digest (and without an area default) means the interface holds a key it never uses, so it keeps forming unauthenticated adjacencies. Fix: ensure the interface's effective type is message-digest — confirm the "Message digest authentication enabled" line in show ip ospf interface.

! Pinpoint an authentication mismatch:
R1# debug ip ospf adj
*Jul  2 14:03:11.204: OSPF-1 ADJ Gi0/0: Rcv pkt from 10.0.12.2 :
    Mismatched Authentication Key - Message Digest Key 1

! Symptom of a bad key: neighbor is absent or never reaches FULL
R1# show ip ospf neighbor
! (no neighbor listed, or State stuck below FULL)

R1# undebug all