From: Karsten Graul <kgraul@linux.ibm.com>
Subject: mlx4: trigger IB events needed by SMC
Patch-mainline: v5.0-rc1
Git-commit: fc6526fba130dcbd496b96a9abf75a9382da95da
References: FATE#325698, LTC#167867, bsc#1113481
Description: net/smc: bugfix and compatibility patches
Symptom: Random hangs in smc processing:
user space application hangs in socket send() or recv() call or
does never get a notification from a select() call.
Missing compatibility to other platforms:
confirm rkey and delete rkey processing is required by the
design, but delete rkey processing is missing. This leads to
protocol failures when communicating with other platforms like
zOS. The SMC-D shutdown signal support is missing, so there is
no detection if the remote peer closed the link group.
Broken administration of available WR send payload buffers due to
a use-after-free condition.
Problem: Misbehaviour regarding the user space api can lead to hang
situations. SMC is not fully compatible to some other platforms
due to missing rkey processing and SMC-D shutdown signal support.
Solution: Fixed protocoll deficiencies by implementing the required rkey
processing. For SMC-D, the cursors are now handled atomically to
handle parallel modifications. The SMC-D shutdown signal is now
processed when received and sent to the remote peer if needed.
Prereq patches are included.
Reproduction: Run SMC on a loaded system against zOS as peer system.
Upstream-Description:
mlx4: trigger IB events needed by SMC
The mlx4 driver does not trigger an IB_EVENT_PORT_ACTIVE when the RoCE
network interface is activated. When SMC determines the RoCE device port
to be used, it checks the port states. This patch triggers IB events for
NETDEV_UP and NETDEV_DOWN.
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
Acked-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Signed-off-by: Karsten Graul <kgraul@linux.ibm.com>
Acked-by: Petr Tesarik <ptesarik@suse.com>
---
drivers/infiniband/hw/mlx4/main.c | 27 +++++++++++++++++++++++++++
drivers/infiniband/hw/mlx4/mlx4_ib.h | 1 +
2 files changed, 28 insertions(+)
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -2440,6 +2440,32 @@ static void mlx4_ib_scan_netdevs(struct
event == NETDEV_UP || event == NETDEV_CHANGE))
update_qps_port = port;
+ if (dev == iboe->netdevs[port - 1] &&
+ (event == NETDEV_UP || event == NETDEV_DOWN)) {
+ enum ib_port_state port_state;
+ struct ib_event ibev = { };
+
+ if (ib_get_cached_port_state(&ibdev->ib_dev, port,
+ &port_state))
+ continue;
+
+ if (event == NETDEV_UP &&
+ (port_state != IB_PORT_ACTIVE ||
+ iboe->last_port_state[port - 1] != IB_PORT_DOWN))
+ continue;
+ if (event == NETDEV_DOWN &&
+ (port_state != IB_PORT_DOWN ||
+ iboe->last_port_state[port - 1] != IB_PORT_ACTIVE))
+ continue;
+ iboe->last_port_state[port - 1] = port_state;
+
+ ibev.device = &ibdev->ib_dev;
+ ibev.element.port_num = port;
+ ibev.event = event == NETDEV_UP ? IB_EVENT_PORT_ACTIVE :
+ IB_EVENT_PORT_ERR;
+ ib_dispatch_event(&ibev);
+ }
+
}
spin_unlock_bh(&iboe->lock);
@@ -2799,6 +2825,7 @@ static void *mlx4_ib_add(struct mlx4_dev
for (i = 0; i < ibdev->num_ports; ++i) {
mutex_init(&ibdev->counters_table[i].mutex);
INIT_LIST_HEAD(&ibdev->counters_table[i].counters_list);
+ iboe->last_port_state[i] = IB_PORT_DOWN;
}
num_req_counters = mlx4_is_bonded(dev) ? 1 : ibdev->num_ports;
--- a/drivers/infiniband/hw/mlx4/mlx4_ib.h
+++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h
@@ -524,6 +524,7 @@ struct mlx4_ib_iboe {
atomic64_t mac[MLX4_MAX_PORTS];
struct notifier_block nb;
struct mlx4_port_gid_table gids[MLX4_MAX_PORTS];
+ enum ib_port_state last_port_state[MLX4_MAX_PORTS];
};
struct pkey_mgt {