rds: don't let RDS shutdown a connection while senders are present

This is the first in a long line of patches that tries to fix races
between RDS connection shutdown and RDS traffic.

Here we are maintaining a count of active senders to make sure
the connection doesn't go away while they are using it.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
diff --git a/net/rds/connection.c b/net/rds/connection.c
index 7e4e9df..9c249f3 100644
--- a/net/rds/connection.c
+++ b/net/rds/connection.c
@@ -148,6 +148,7 @@
 
 	spin_lock_init(&conn->c_send_lock);
 	atomic_set(&conn->c_send_generation, 1);
+	atomic_set(&conn->c_senders, 0);
 	INIT_LIST_HEAD(&conn->c_send_queue);
 	INIT_LIST_HEAD(&conn->c_retrans);
 
@@ -276,6 +277,12 @@
 		spin_lock_irq(&conn->c_send_lock);
 		spin_unlock_irq(&conn->c_send_lock);
 
+		while(atomic_read(&conn->c_senders)) {
+			schedule_timeout(1);
+			spin_lock_irq(&conn->c_send_lock);
+			spin_unlock_irq(&conn->c_send_lock);
+		}
+
 		conn->c_trans->conn_shutdown(conn);
 		rds_conn_reset(conn);