Advantages of heaps over binary trees? Illustration with the Dijkstra algorithm

Advantages of heaps over binary trees? Illustration with the Dijkstra
algorithm

One of the standard implementation of the Dijkstra algorithm uses a heap
to store distances from the starting node S to all unexplored nodes. The
argument for using a heap is that we can efficiently pop the minimum
distance from it, in O(log n). However, to maintain the invariant of the
algorithm, one also needs to update some of the distances in the heap.
This involves:
popping non-min elements from the heaps
computing the updated distances
inserting them back into the heap
I understand that popping non-min elements from a heap can be done in
O(log n) if one knows the location of that element in the heap. However, I
fail to understand how one can know this location in the case of the
Dijkstra algorithm. It sounds like a binary search tree would be more
appropriate.
More generally, my understanding is that the only thing that a heap can do
better than a balanced binary search tree is to access (without removing)
the min element. Is my understanding correct?