m0pt0pmatt17 | 20 Feb 22:37 2015
Picon

Changes to iSCSI device are not consistent across network

Hey guys,

Forgive me, but I'm super new to this.

I have two CentOS 7 nodes. I'm using LIO to export a sparse file over iSCSI.

The sparse file was created as a LIO FILEIO with write-back disabled (write-through)
In targetcli, I create a LUN on my iSCSI frontend

I formatted the sparse file to have an EXT4 filesystem.

On both the target node and the initiator node, I can initiate a iSCSI session (iscsiadm -m node --login), mount the device, and read and write to it.

However, changes to the device are not consistent across the network until i logout of the iSCSI session. (iscsiadm -m node --logout) (both nodes have to logout. The first logout writes the changes, and the second one refreshes them)

Somewhere, caching is occurring, but I'm not sure where.

Just in case you're curious, my use case is to have multiple nodes write to the same remote disk (or file) in parallel. 

Any direction or advice would be great. Thank you.

-Matt

--
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To post to this group, send email to open-iscsi-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
Mike Christie | 23 Feb 04:25 2015
Picon

Re: [Lsf] It's time to put together the schedule

Dropping lsf for iscsi list.

On 2/22/15, 12:12 PM, Sagi Grimberg wrote:
> On 2/22/2015 7:51 PM, James Bottomley wrote:
>> On Sun, 2015-02-22 at 18:57 +0200, Sagi Grimberg wrote:
>>> On 2/22/2015 6:32 PM, James Bottomley wrote:
>>>
>>>>> Hey All,
>>>>>
>>>>> I'm also interested in talking about block/scsi-mq. Moreover, I was
>>>>> hoping we can talk about iSCSI in the context of scsi-mq. We
>>>>> started the
>>>>> discussion on the list
>>>>> (http://marc.info/?l=linux-scsi&w=2&r=2&s=iSCSI+MQ+adoption+via+MCS+discussion&q=b)
>>>>>
>>>>> and I'd like us to come
>>>>> to an agreement on this.
>>>>>
>>>>> What I would like to talk about is:
>>>>> - How will we implement multiple submission queues?
>>>>>      MCS or multiple sessions - pros/cons.
>>>>>
>>>>> - Fitting iSCSI sockets to HW contexts is trivial, but getting
>>>>>      the completions to steer to the correct CPU is something we
>>>>> may want
>>>>>      to discuss. Do we want to control that? or do we want to let the
>>>>>      networking stack to do handle it like any other socket (RFS,
>>>>> XPS ...)
>>>>>
>>>>> I think it might be worth do dedicate a session for this if we have an
>>>>> open slot.
>>>>
>>>> We have a few open slots, yes.  I thought this was largely a solved
>>>> problem.  If not, I can add it to the storage track.
>>>
>>> IMO this thread didn't come to any final conclusions. Lets see if this
>>> session is interesting to anyone else...
>>
>> OK, let's see who pipes up.  I have to say I have institutional bias
>> against MC/S since it seems to me to try to take us back to the bad old
>> days where every "enterprise" SCSI driver saw value add in implementing
>> its own version of multi-path.  Trying to get them all removed in favour
>> of dm-multipath was a long hard battle I don't really want to revisit.
>>
>> On the other hand, if there's a real problem with dm-multipath and
>> scsi-mq which MC/S solves, I'd also like to know about it ... although
>> probably from a "how can we fix dm-multipath" point of view.
>
> I don't think anyone is actually considering implementing MCS to do any
> type of HA or load-balancing but rather just make it fit the multi-queue
> schema.
>
> We can do multiple sessions but then we either make iSCSI expose a
> scsi_host per port (not per session like today) or create some kind of
> abstraction layer to manage multiple sessions (like srp_target_port
> which holds several srp_rdma_ch).
>
> I think different folks might have different views on this...
>

Sorry about that. I thought I sent this to you. It must have got lost. 
Attached is the kernel changes for session based mq support (patch made 
over linus's tree). It just has the core changes only. I did not yet 
modify iscsi_tcp/libiscsi to bind our threads and sockets to cpus or 
deal with the networking.

I was waiting to hear back from you about how it could work for ib_iser 
and was waiting to hear back from some hw vendors about how they are 
going to support mq in their hw/fw. So, the queue_id that gets passed in 
is really generic at this point and can map to whatever the transport 
driver wants to map it to (actual hw/fw queue for offload, msix/cpu 
combo like srp for iser, cpu and something else for iscsi_tcp, etc).

I just hit a bug in the userspace code. Will send that later.

But yeah, just added a new struct between the host and sessions.

-- 
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe@...
To post to this group, send email to open-iscsi@...
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.c b/drivers/infiniband/ulp/iser/iscsi_iser.c
index 6a594aa..b0d2e73 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.c
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.c
 <at>  <at>  -410,13 +410,14  <at>  <at>  iscsi_iser_check_protection(struct iscsi_task *task, sector_t *sector)
  * iscsi_iser_conn_create() - create a new iscsi-iser connection
  *  <at> cls_session: iscsi class connection
  *  <at> conn_idx:    connection index within the session (for MCS)
+ *  <at> queue_idx:   mq queue id
  *
  * Return: iscsi_cls_conn when iscsi_conn_setup succeeds or NULL
  *         otherwise.
  */
 static struct iscsi_cls_conn *
 iscsi_iser_conn_create(struct iscsi_cls_session *cls_session,
-		       uint32_t conn_idx)
+		       uint32_t conn_idx, uint32_t queue_id)
 {
 	struct iscsi_conn *conn;
 	struct iscsi_cls_conn *cls_conn;
 <at>  <at>  -564,7 +565,7  <at>  <at>  iscsi_iser_conn_stop(struct iscsi_cls_conn *cls_conn, int flag)
 static void
 iscsi_iser_session_destroy(struct iscsi_cls_session *cls_session)
 {
-	struct Scsi_Host *shost = iscsi_session_to_shost(cls_session);
+	struct Scsi_Host *shost = dev_to_shost(&cls_session->dev);
 
 	iscsi_session_teardown(cls_session);
 	iscsi_host_remove(shost);
 <at>  <at>  -585,18 +586,21  <at>  <at>  iser_dif_prot_caps(int prot_caps)
 
 /**
  * iscsi_iser_session_create() - create an iscsi-iser session
+ *  <at> grp:            grp to add session to if mq is enabled
  *  <at> ep:             iscsi end-point handle
  *  <at> cmds_max:       maximum commands in this session
  *  <at> qdepth:         session command queue depth
  *  <at> initial_cmdsn:  initiator command sequnce number
+ *  <at> queue_id:       mq queue id
  *
  * Allocates and adds a scsi host, expose DIF supprot if
  * exists, and sets up an iscsi session.
  */
 static struct iscsi_cls_session *
-iscsi_iser_session_create(struct iscsi_endpoint *ep,
+iscsi_iser_session_create(struct iscsi_session_grp *grp,
+			  struct iscsi_endpoint *ep,
 			  uint16_t cmds_max, uint16_t qdepth,
-			  uint32_t initial_cmdsn)
+			  uint32_t initial_cmdsn, uint32_t queue_id)
 {
 	struct iscsi_cls_session *cls_session;
 	struct iscsi_session *session;
 <at>  <at>  -658,10 +662,10  <at>  <at>  iscsi_iser_session_create(struct iscsi_endpoint *ep,
 		cmds_max = max_cmds;
 	}
 
-	cls_session = iscsi_session_setup(&iscsi_iser_transport, shost,
+	cls_session = iscsi_session_setup(&iscsi_iser_transport, shost, grp,
 					  cmds_max, 0,
 					  sizeof(struct iscsi_iser_task),
-					  initial_cmdsn, 0);
+					  initial_cmdsn, queue_id, 0);
 	if (!cls_session)
 		goto remove_host;
 	session = cls_session->dd_data;
 <at>  <at>  -781,6 +785,7  <at>  <at>  static int iscsi_iser_get_ep_param(struct iscsi_endpoint *ep,
  *  <at> shost:          scsi_host
  *  <at> dst_addr:       destination address
  *  <at> non-blocking:   indicate if routine can block
+ *  <at> queue_id:       mq queue id
  *
  * Allocate an iscsi endpoint, an iser_conn structure and bind them.
  * After that start RDMA connection establishment via rdma_cm. We
 <at>  <at>  -793,7 +798,7  <at>  <at>  static int iscsi_iser_get_ep_param(struct iscsi_endpoint *ep,
  */
 static struct iscsi_endpoint *
 iscsi_iser_ep_connect(struct Scsi_Host *shost, struct sockaddr *dst_addr,
-		      int non_blocking)
+		      int non_blocking, uint32_t queue_id)
 {
 	int err;
 	struct iser_conn *iser_conn;
diff --git a/drivers/scsi/be2iscsi/be_cmds.c b/drivers/scsi/be2iscsi/be_cmds.c
index 80d97f3..c9a57ff 100644
--- a/drivers/scsi/be2iscsi/be_cmds.c
+++ b/drivers/scsi/be2iscsi/be_cmds.c
 <at>  <at>  -421,7 +421,7  <at>  <at>  static struct be_mcc_compl *be_mcc_compl_get(struct beiscsi_hba *phba)
  **/
 void be2iscsi_fail_session(struct iscsi_cls_session *cls_session)
 {
-	struct Scsi_Host *shost = iscsi_session_to_shost(cls_session);
+	struct Scsi_Host *shost = dev_to_shost(&cls_session->dev);
 	struct beiscsi_hba *phba = iscsi_host_priv(shost);
 	uint32_t iscsi_err_flag;
 
diff --git a/drivers/scsi/be2iscsi/be_iscsi.c b/drivers/scsi/be2iscsi/be_iscsi.c
index b7391a3..ebfac73 100644
--- a/drivers/scsi/be2iscsi/be_iscsi.c
+++ b/drivers/scsi/be2iscsi/be_iscsi.c
 <at>  <at>  -33,14 +33,19  <at>  <at>  extern struct iscsi_transport beiscsi_iscsi_transport;
 
 /**
  * beiscsi_session_create - creates a new iscsi session
+ *  <at> grp: grp to add session to if mq is enabled
+ *  <at> ep: endpoint handle
  *  <at> cmds_max: max commands supported
  *  <at> qdepth: max queue depth supported
  *  <at> initial_cmdsn: initial iscsi CMDSN
+ *  <at> queue_id: mq queue id
  */
-struct iscsi_cls_session *beiscsi_session_create(struct iscsi_endpoint *ep,
+struct iscsi_cls_session *beiscsi_session_create(struct iscsi_session_grp *grp,
+						 struct iscsi_endpoint *ep,
 						 u16 cmds_max,
 						 u16 qdepth,
-						 u32 initial_cmdsn)
+						 u32 initial_cmdsn,
+						 u32 queue_id)
 {
 	struct Scsi_Host *shost;
 	struct beiscsi_endpoint *beiscsi_ep;
 <at>  <at>  -81,10 +86,11  <at>  <at>  struct iscsi_cls_session *beiscsi_session_create(struct iscsi_endpoint *ep,
 
 	shost = phba->shost;
 	cls_session = iscsi_session_setup(&beiscsi_iscsi_transport,
-					  shost, cmds_max,
+					  shost, grp, cmds_max,
 					  sizeof(*beiscsi_sess),
 					  sizeof(*io_task),
-					  initial_cmdsn, ISCSI_MAX_TARGET);
+					  initial_cmdsn, queue_id,
+					  ISCSI_MAX_TARGET);
 	if (!cls_session)
 		return NULL;
 	sess = cls_session->dd_data;
 <at>  <at>  -123,9 +129,12  <at>  <at>  void beiscsi_session_destroy(struct iscsi_cls_session *cls_session)
  * beiscsi_conn_create - create an instance of iscsi connection
  *  <at> cls_session: ptr to iscsi_cls_session
  *  <at> cid: iscsi cid
+ *  <at> queue_id: mq queue id
+ *
  */
 struct iscsi_cls_conn *
-beiscsi_conn_create(struct iscsi_cls_session *cls_session, u32 cid)
+beiscsi_conn_create(struct iscsi_cls_session *cls_session, u32 cid,
+		    u32 queue_id)
 {
 	struct beiscsi_hba *phba;
 	struct Scsi_Host *shost;
 <at>  <at>  -135,7 +144,7  <at>  <at>  beiscsi_conn_create(struct iscsi_cls_session *cls_session, u32 cid)
 	struct iscsi_session *sess;
 	struct beiscsi_session *beiscsi_sess;
 
-	shost = iscsi_session_to_shost(cls_session);
+	shost = dev_to_shost(&cls_session->dev);
 	phba = iscsi_host_priv(shost);
 
 	beiscsi_log(phba, KERN_INFO, BEISCSI_LOG_CONFIG,
 <at>  <at>  -198,7 +207,7  <at>  <at>  int beiscsi_conn_bind(struct iscsi_cls_session *cls_session,
 {
 	struct iscsi_conn *conn = cls_conn->dd_data;
 	struct beiscsi_conn *beiscsi_conn = conn->dd_data;
-	struct Scsi_Host *shost = iscsi_session_to_shost(cls_session);
+	struct Scsi_Host *shost = dev_to_shost(&cls_session->dev);
 	struct beiscsi_hba *phba = iscsi_host_priv(shost);
 	struct hwi_controller *phwi_ctrlr = phba->phwi_ctrlr;
 	struct hwi_wrb_context *pwrb_context;
 <at>  <at>  -1190,12 +1199,13  <at>  <at>  static int beiscsi_open_conn(struct iscsi_endpoint *ep,
  *  <at> scsi_host: Pointer to scsi_host structure
  *  <at> dst_addr: The IP address of Target
  *  <at> non_blocking: blocking or non-blocking call
+ *  <at> queue_id: mq queue id
  *
  * This routines first asks chip to create a connection and then allocates an EP
  */
 struct iscsi_endpoint *
 beiscsi_ep_connect(struct Scsi_Host *shost, struct sockaddr *dst_addr,
-		   int non_blocking)
+		   int non_blocking, u32 queue_id)
 {
 	struct beiscsi_hba *phba;
 	struct beiscsi_endpoint *beiscsi_ep;
diff --git a/drivers/scsi/be2iscsi/be_iscsi.h b/drivers/scsi/be2iscsi/be_iscsi.h
index e0b3b2d..cd2187d 100644
--- a/drivers/scsi/be2iscsi/be_iscsi.h
+++ b/drivers/scsi/be2iscsi/be_iscsi.h
 <at>  <at>  -50,15 +50,18  <at>  <at>  void beiscsi_offload_iscsi(struct beiscsi_hba *phba, struct iscsi_conn *conn,
 			   struct beiscsi_conn *beiscsi_conn,
 			   unsigned int fw_handle);
 
-struct iscsi_cls_session *beiscsi_session_create(struct iscsi_endpoint *ep,
+struct iscsi_cls_session *beiscsi_session_create(struct iscsi_session_grp *grp,
+						 struct iscsi_endpoint *ep,
 						 uint16_t cmds_max,
 						 uint16_t qdepth,
-						 uint32_t initial_cmdsn);
+						 uint32_t initial_cmdsn,
+						 uint32_t queue_id);
 
 void beiscsi_session_destroy(struct iscsi_cls_session *cls_session);
 
 struct iscsi_cls_conn *beiscsi_conn_create(struct iscsi_cls_session
-					   *cls_session, uint32_t cid);
+					   *cls_session, uint32_t cid,
+					   uint32_t queue_id);
 
 int beiscsi_conn_bind(struct iscsi_cls_session *cls_session,
 		      struct iscsi_cls_conn *cls_conn,
 <at>  <at>  -79,7 +82,7  <at>  <at>  int beiscsi_conn_start(struct iscsi_cls_conn *cls_conn);
 
 struct iscsi_endpoint *beiscsi_ep_connect(struct Scsi_Host *shost,
 					  struct sockaddr *dst_addr,
-					  int non_blocking);
+					  int non_blocking, uint32_t queue_id);
 
 int beiscsi_ep_poll(struct iscsi_endpoint *ep, int timeout_ms);
 
diff --git a/drivers/scsi/be2iscsi/be_main.c b/drivers/scsi/be2iscsi/be_main.c
index f319340..8f8d4d1 100644
--- a/drivers/scsi/be2iscsi/be_main.c
+++ b/drivers/scsi/be2iscsi/be_main.c
 <at>  <at>  -218,7 +218,6  <at>  <at>  static int beiscsi_slave_configure(struct scsi_device *sdev)
 
 static int beiscsi_eh_abort(struct scsi_cmnd *sc)
 {
-	struct iscsi_cls_session *cls_session;
 	struct iscsi_task *aborted_task = (struct iscsi_task *)sc->SCp.ptr;
 	struct beiscsi_io_task *aborted_io_task;
 	struct iscsi_conn *conn;
 <at>  <at>  -230,9 +229,7  <at>  <at>  static int beiscsi_eh_abort(struct scsi_cmnd *sc)
 	unsigned int cid, tag, num_invalidate;
 	int rc;
 
-	cls_session = starget_to_session(scsi_target(sc->device));
-	session = cls_session->dd_data;
-
+	session = scsi_cmd_to_session(sc);
 	spin_lock_bh(&session->frwd_lock);
 	if (!aborted_task || !aborted_task->sc) {
 		/* we raced */
 <at>  <at>  -302,15 +299,13  <at>  <at>  static int beiscsi_eh_device_reset(struct scsi_cmnd *sc)
 	struct beiscsi_conn *beiscsi_conn;
 	struct beiscsi_hba *phba;
 	struct iscsi_session *session;
-	struct iscsi_cls_session *cls_session;
 	struct invalidate_command_table *inv_tbl;
 	struct be_dma_mem nonemb_cmd;
 	unsigned int cid, tag, i, num_invalidate;
 	int rc;
 
 	/* invalidate iocbs */
-	cls_session = starget_to_session(scsi_target(sc->device));
-	session = cls_session->dd_data;
+	session = scsi_cmd_to_session(sc);
 	spin_lock_bh(&session->frwd_lock);
 	if (!session->leadconn || session->state != ISCSI_STATE_LOGGED_IN) {
 		spin_unlock_bh(&session->frwd_lock);
diff --git a/drivers/scsi/bnx2i/bnx2i_iscsi.c b/drivers/scsi/bnx2i/bnx2i_iscsi.c
index e53078d..55d2aa4 100644
--- a/drivers/scsi/bnx2i/bnx2i_iscsi.c
+++ b/drivers/scsi/bnx2i/bnx2i_iscsi.c
 <at>  <at>  -1222,7 +1222,7  <at>  <at>  static int bnx2i_task_xmit(struct iscsi_task *task)
 {
 	struct iscsi_conn *conn = task->conn;
 	struct iscsi_session *session = conn->session;
-	struct Scsi_Host *shost = iscsi_session_to_shost(session->cls_session);
+	struct Scsi_Host *shost = dev_to_shost(&session->cls_session->dev);
 	struct bnx2i_hba *hba = iscsi_host_priv(shost);
 	struct bnx2i_conn *bnx2i_conn = conn->dd_data;
 	struct scsi_cmnd *sc = task->sc;
 <at>  <at>  -1275,16 +1275,19  <at>  <at>  static int bnx2i_task_xmit(struct iscsi_task *task)
 
 /**
  * bnx2i_session_create - create a new iscsi session
+ *  <at> grp:		grp to add session to if mq is enabled
+ *  <at> ep:			endpoint handle
  *  <at> cmds_max:		max commands supported
  *  <at> qdepth:		scsi queue depth to support
  *  <at> initial_cmdsn:	initial iscsi CMDSN to be used for this session
+ *  <at> queue_id:		mq queue id
  *
  * Creates a new iSCSI session instance on given device.
  */
 static struct iscsi_cls_session *
-bnx2i_session_create(struct iscsi_endpoint *ep,
+bnx2i_session_create(struct iscsi_session_grp *grp, struct iscsi_endpoint *ep,
 		     uint16_t cmds_max, uint16_t qdepth,
-		     uint32_t initial_cmdsn)
+		     uint32_t initial_cmdsn, uint32_t queue_id)
 {
 	struct Scsi_Host *shost;
 	struct iscsi_cls_session *cls_session;
 <at>  <at>  -1311,9 +1314,10  <at>  <at>  bnx2i_session_create(struct iscsi_endpoint *ep,
 	else if (cmds_max < BNX2I_SQ_WQES_MIN)
 		cmds_max = BNX2I_SQ_WQES_MIN;
 
-	cls_session = iscsi_session_setup(&bnx2i_iscsi_transport, shost,
+	cls_session = iscsi_session_setup(&bnx2i_iscsi_transport, shost, grp,
 					  cmds_max, 0, sizeof(struct bnx2i_cmd),
-					  initial_cmdsn, ISCSI_MAX_TARGET);
+					  initial_cmdsn, queue_id,
+					  ISCSI_MAX_TARGET);
 	if (!cls_session)
 		return NULL;
 
 <at>  <at>  -1337,7 +1341,7  <at>  <at>  session_teardown:
 static void bnx2i_session_destroy(struct iscsi_cls_session *cls_session)
 {
 	struct iscsi_session *session = cls_session->dd_data;
-	struct Scsi_Host *shost = iscsi_session_to_shost(cls_session);
+	struct Scsi_Host *shost = dev_to_shost(&cls_session->dev);
 	struct bnx2i_hba *hba = iscsi_host_priv(shost);
 
 	bnx2i_destroy_cmd_pool(hba, session);
 <at>  <at>  -1349,13 +1353,15  <at>  <at>  static void bnx2i_session_destroy(struct iscsi_cls_session *cls_session)
  * bnx2i_conn_create - create iscsi connection instance
  *  <at> cls_session:	pointer to iscsi cls session
  *  <at> cid:		iscsi cid as per rfc (not NX2's CID terminology)
+ *  <at> queue_id:		mq queue id
  *
  * Creates a new iSCSI connection instance for a given session
  */
 static struct iscsi_cls_conn *
-bnx2i_conn_create(struct iscsi_cls_session *cls_session, uint32_t cid)
+bnx2i_conn_create(struct iscsi_cls_session *cls_session, uint32_t cid,
+		  uint32_t queue_id)
 {
-	struct Scsi_Host *shost = iscsi_session_to_shost(cls_session);
+	struct Scsi_Host *shost = dev_to_shost(&cls_session->dev);
 	struct bnx2i_hba *hba = iscsi_host_priv(shost);
 	struct bnx2i_conn *bnx2i_conn;
 	struct iscsi_cls_conn *cls_conn;
 <at>  <at>  -1408,7 +1414,7  <at>  <at>  static int bnx2i_conn_bind(struct iscsi_cls_session *cls_session,
 {
 	struct iscsi_conn *conn = cls_conn->dd_data;
 	struct bnx2i_conn *bnx2i_conn = conn->dd_data;
-	struct Scsi_Host *shost = iscsi_session_to_shost(cls_session);
+	struct Scsi_Host *shost = dev_to_shost(&cls_session->dev);
 	struct bnx2i_hba *hba = iscsi_host_priv(shost);
 	struct bnx2i_endpoint *bnx2i_ep;
 	struct iscsi_endpoint *ep;
 <at>  <at>  -1480,7 +1486,7  <at>  <at>  static void bnx2i_conn_destroy(struct iscsi_cls_conn *cls_conn)
 	unsigned cpu = 0;
 	struct bnx2i_percpu_s *p;
 
-	shost = iscsi_session_to_shost(iscsi_conn_to_session(cls_conn));
+	shost = dev_to_shost(&cls_conn->dev);
 	hba = iscsi_host_priv(shost);
 
 	bnx2i_conn_free_login_resources(hba, bnx2i_conn);
 <at>  <at>  -1761,6 +1767,7  <at>  <at>  static int bnx2i_tear_down_conn(struct bnx2i_hba *hba,
  *  <at> shost:		scsi host
  *  <at> dst_addr:		target IP address
  *  <at> non_blocking:	blocking or non-blocking call
+ *  <at> queue_id:		mq queue id
  *
  * this routine initiates the TCP/IP connection by invoking Option-2 i/f
  *	with l5_core and the CNIC. This is a multi-step process of resolving
 <at>  <at>  -1770,7 +1777,8  <at>  <at>  static int bnx2i_tear_down_conn(struct bnx2i_hba *hba,
  */
 static struct iscsi_endpoint *bnx2i_ep_connect(struct Scsi_Host *shost,
 					       struct sockaddr *dst_addr,
-					       int non_blocking)
+					       int non_blocking,
+					       uint32_t queue_id)
 {
 	u32 iscsi_cid = BNX2I_CID_RESERVED;
 	struct sockaddr_in *desti = (struct sockaddr_in *) dst_addr;
diff --git a/drivers/scsi/cxgbi/libcxgbi.c b/drivers/scsi/cxgbi/libcxgbi.c
index eb58afc..96d2196 100644
--- a/drivers/scsi/cxgbi/libcxgbi.c
+++ b/drivers/scsi/cxgbi/libcxgbi.c
 <at>  <at>  -2492,7 +2492,7  <at>  <at>  int cxgbi_get_ep_param(struct iscsi_endpoint *ep, enum iscsi_param param,
 EXPORT_SYMBOL_GPL(cxgbi_get_ep_param);
 
 struct iscsi_cls_conn *
-cxgbi_create_conn(struct iscsi_cls_session *cls_session, u32 cid)
+cxgbi_create_conn(struct iscsi_cls_session *cls_session, u32 cid, u32 queue_id)
 {
 	struct iscsi_cls_conn *cls_conn;
 	struct iscsi_conn *conn;
 <at>  <at>  -2566,9 +2566,10  <at>  <at>  int cxgbi_bind_conn(struct iscsi_cls_session *cls_session,
 }
 EXPORT_SYMBOL_GPL(cxgbi_bind_conn);
 
-struct iscsi_cls_session *cxgbi_create_session(struct iscsi_endpoint *ep,
+struct iscsi_cls_session *cxgbi_create_session(struct iscsi_session_grp *grp,
+						struct iscsi_endpoint *ep,
 						u16 cmds_max, u16 qdepth,
-						u32 initial_cmdsn)
+						u32 initial_cmdsn, u32 queue_id)
 {
 	struct cxgbi_endpoint *cep;
 	struct cxgbi_hba *chba;
 <at>  <at>  -2587,11 +2588,12  <at>  <at>  struct iscsi_cls_session *cxgbi_create_session(struct iscsi_endpoint *ep,
 
 	BUG_ON(chba != iscsi_host_priv(shost));
 
-	cls_session = iscsi_session_setup(chba->cdev->itp, shost,
+	cls_session = iscsi_session_setup(chba->cdev->itp, shost, grp,
 					cmds_max, 0,
 					sizeof(struct iscsi_tcp_task) +
 					sizeof(struct cxgbi_task_data),
-					initial_cmdsn, ISCSI_MAX_TARGET);
+					initial_cmdsn, queue_id,
+					ISCSI_MAX_TARGET);
 	if (!cls_session)
 		return NULL;
 
 <at>  <at>  -2697,7 +2699,7  <at>  <at>  EXPORT_SYMBOL_GPL(cxgbi_get_host_param);
 
 struct iscsi_endpoint *cxgbi_ep_connect(struct Scsi_Host *shost,
 					struct sockaddr *dst_addr,
-					int non_blocking)
+					int non_blocking, u32 queue_id)
 {
 	struct iscsi_endpoint *ep;
 	struct cxgbi_endpoint *cep;
diff --git a/drivers/scsi/cxgbi/libcxgbi.h b/drivers/scsi/cxgbi/libcxgbi.h
index aba1af7..3e53e14 100644
--- a/drivers/scsi/cxgbi/libcxgbi.h
+++ b/drivers/scsi/cxgbi/libcxgbi.h
 <at>  <at>  -729,17 +729,17  <at>  <at>  void cxgbi_get_conn_stats(struct iscsi_cls_conn *, struct iscsi_stats *);
 int cxgbi_set_conn_param(struct iscsi_cls_conn *,
 			enum iscsi_param, char *, int);
 int cxgbi_get_ep_param(struct iscsi_endpoint *ep, enum iscsi_param, char *);
-struct iscsi_cls_conn *cxgbi_create_conn(struct iscsi_cls_session *, u32);
+struct iscsi_cls_conn *cxgbi_create_conn(struct iscsi_cls_session *, u32, u32);
 int cxgbi_bind_conn(struct iscsi_cls_session *,
 			struct iscsi_cls_conn *, u64, int);
 void cxgbi_destroy_session(struct iscsi_cls_session *);
-struct iscsi_cls_session *cxgbi_create_session(struct iscsi_endpoint *,
-			u16, u16, u32);
+struct iscsi_cls_session *cxgbi_create_session(struct iscsi_session_grp *,
+			struct iscsi_endpoint *, u16, u16, u32, u32);
 int cxgbi_set_host_param(struct Scsi_Host *,
 			enum iscsi_host_param, char *, int);
 int cxgbi_get_host_param(struct Scsi_Host *, enum iscsi_host_param, char *);
 struct iscsi_endpoint *cxgbi_ep_connect(struct Scsi_Host *,
-			struct sockaddr *, int);
+			struct sockaddr *, int, u32);
 int cxgbi_ep_poll(struct iscsi_endpoint *, int);
 void cxgbi_ep_disconnect(struct iscsi_endpoint *);
 
diff --git a/drivers/scsi/iscsi_tcp.c b/drivers/scsi/iscsi_tcp.c
index 0b8af18..d0626a4 100644
--- a/drivers/scsi/iscsi_tcp.c
+++ b/drivers/scsi/iscsi_tcp.c
 <at>  <at>  -537,7 +537,7  <at>  <at>  static int iscsi_sw_tcp_pdu_alloc(struct iscsi_task *task, uint8_t opcode)
 
 static struct iscsi_cls_conn *
 iscsi_sw_tcp_conn_create(struct iscsi_cls_session *cls_session,
-			 uint32_t conn_idx)
+			 uint32_t conn_idx, uint32_t queue_idx)
 {
 	struct iscsi_conn *conn;
 	struct iscsi_cls_conn *cls_conn;
 <at>  <at>  -815,9 +815,67  <at>  <at>  iscsi_sw_tcp_conn_get_stats(struct iscsi_cls_conn *cls_conn,
 	iscsi_tcp_conn_get_stats(cls_conn, stats);
 }
 
+static struct Scsi_Host *iscsi_sw_tcp_create_host(void)
+{
+	struct Scsi_Host *shost;
+
+	shost = iscsi_host_alloc(&iscsi_sw_tcp_sht,
+				 sizeof(struct iscsi_sw_tcp_host), 1);
+	if (!shost)
+		return shost;
+
+	shost->transportt = iscsi_sw_tcp_scsi_transport;
+	shost->max_lun = iscsi_max_lun;
+	shost->max_id = 0;
+	shost->max_channel = 0;
+	shost->max_cmd_len = SCSI_MAX_VARLEN_CDB_SIZE;
+
+	if (iscsi_host_add(shost, NULL))
+		goto free_host;
+
+	return shost;
+free_host:
+	iscsi_host_free(shost);
+	return NULL;
+}
+
+void iscsi_sw_tcp_session_grp_destroy(struct iscsi_session_grp *grp)
+{
+	struct Scsi_Host *shost = dev_to_shost(&grp->dev);
+
+	iscsi_host_remove(shost);
+	iscsi_host_free(shost);
+	iscsi_destroy_session_grp(grp);
+}
+
+static struct iscsi_session_grp *
+iscsi_sw_tcp_session_grp_create(uint32_t host_no)
+{
+	struct Scsi_Host *shost;
+	struct iscsi_session_grp *grp;
+
+	shost = iscsi_sw_tcp_create_host();
+	if (!shost)
+		goto fail;
+
+	grp = iscsi_create_session_grp(shost);
+	if (!grp)
+		goto destroy_host;
+
+	return grp;
+
+destroy_host:
+	iscsi_host_remove(shost);
+	iscsi_host_free(shost);
+fail:
+	return NULL;
+}
+
 static struct iscsi_cls_session *
-iscsi_sw_tcp_session_create(struct iscsi_endpoint *ep, uint16_t cmds_max,
-			    uint16_t qdepth, uint32_t initial_cmdsn)
+iscsi_sw_tcp_session_create(struct iscsi_session_grp *grp,
+			    struct iscsi_endpoint *ep, uint16_t cmds_max,
+			    uint16_t qdepth, uint32_t initial_cmdsn,
+			    uint32_t queue_id)
 {
 	struct iscsi_cls_session *cls_session;
 	struct iscsi_session *session;
 <at>  <at>  -829,25 +887,28  <at>  <at>  iscsi_sw_tcp_session_create(struct iscsi_endpoint *ep, uint16_t cmds_max,
 		return NULL;
 	}
 
-	shost = iscsi_host_alloc(&iscsi_sw_tcp_sht,
-				 sizeof(struct iscsi_sw_tcp_host), 1);
+	/*
+	 * MQ enabled tools do a host per session group. Non-mq tools expect
+	 * host per session.
+	 */
+	if (grp) {
+		shost = dev_to_shost(&grp->dev);
+if (!shost)
+	printk(KERN_ERR "could not find host\n");
+	} else {
+printk(KERN_ERR "doh\n");
+		shost = iscsi_sw_tcp_create_host();
+	}
+
 	if (!shost)
 		return NULL;
-	shost->transportt = iscsi_sw_tcp_scsi_transport;
 	shost->cmd_per_lun = qdepth;
-	shost->max_lun = iscsi_max_lun;
-	shost->max_id = 0;
-	shost->max_channel = 0;
-	shost->max_cmd_len = SCSI_MAX_VARLEN_CDB_SIZE;
-
-	if (iscsi_host_add(shost, NULL))
-		goto free_host;
 
 	cls_session = iscsi_session_setup(&iscsi_sw_tcp_transport, shost,
-					  cmds_max, 0,
+					  grp, cmds_max, 0,
 					  sizeof(struct iscsi_tcp_task) +
 					  sizeof(struct iscsi_sw_tcp_hdrbuf),
-					  initial_cmdsn, 0);
+					  initial_cmdsn, queue_id, 0);
 	if (!cls_session)
 		goto remove_host;
 	session = cls_session->dd_data;
 <at>  <at>  -862,21 +923,28  <at>  <at>  iscsi_sw_tcp_session_create(struct iscsi_endpoint *ep, uint16_t cmds_max,
 remove_session:
 	iscsi_session_teardown(cls_session);
 remove_host:
-	iscsi_host_remove(shost);
-free_host:
-	iscsi_host_free(shost);
+	if (!grp) {
+		iscsi_host_remove(shost);
+		iscsi_host_free(shost);
+	}
 	return NULL;
 }
 
 static void iscsi_sw_tcp_session_destroy(struct iscsi_cls_session *cls_session)
 {
-	struct Scsi_Host *shost = iscsi_session_to_shost(cls_session);
+	struct Scsi_Host *shost = dev_to_shost(&cls_session->dev);
+	bool in_group = false;
+
+	if (cls_session->grp)
+		in_group = true;
 
 	iscsi_tcp_r2tpool_free(cls_session->dd_data);
 	iscsi_session_teardown(cls_session);
 
-	iscsi_host_remove(shost);
-	iscsi_host_free(shost);
+	if (!in_group) {
+		iscsi_host_remove(shost);
+		iscsi_host_free(shost);
+	}
 }
 
 static umode_t iscsi_sw_tcp_attr_is_visible(int param_type, int param)
 <at>  <at>  -975,6 +1043,8  <at>  <at>  static struct iscsi_transport iscsi_sw_tcp_transport = {
 	.caps			= CAP_RECOVERY_L0 | CAP_MULTI_R2T | CAP_HDRDGST
 				  | CAP_DATADGST,
 	/* session management */
+	.create_session_grp	= iscsi_sw_tcp_session_grp_create,
+	.destroy_session_grp	= iscsi_sw_tcp_session_grp_destroy,
 	.create_session		= iscsi_sw_tcp_session_create,
 	.destroy_session	= iscsi_sw_tcp_session_destroy,
 	/* connection management */
diff --git a/drivers/scsi/libiscsi.c b/drivers/scsi/libiscsi.c
index 8053f24..c0958db 100644
--- a/drivers/scsi/libiscsi.c
+++ b/drivers/scsi/libiscsi.c
 <at>  <at>  -27,6 +27,10  <at>  <at> 
 #include <linux/log2.h>
 #include <linux/slab.h>
 #include <linux/module.h>
+#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
+/* tmp hack for look up. Next patch will use tagging lookup or add proper callout and remove this */
+#include <../../block/blk-mq.h>
 #include <asm/unaligned.h>
 #include <net/tcp.h>
 #include <scsi/scsi_cmnd.h>
 <at>  <at>  -1628,6 +1632,28  <at>  <at>  static inline struct iscsi_task *iscsi_alloc_task(struct iscsi_conn *conn,
 	return task;
 }
 
+struct iscsi_session *scsi_cmd_to_session(struct scsi_cmnd *sc)
+{
+	unsigned int queue_id;
+	struct iscsi_session_grp *grp;
+	struct iscsi_cls_session *cls_session;
+
+	if (sc->request->mq_ctx) {
+		/* temp hack - we should not be digging into these structs. Add callout/helpers so LLD can store session in hctx->driver_data, or add support for mq tagging and use that look up like srp. Next patch will fix */
+
+		queue_id = sc->request->q->mq_map[sc->request->mq_ctx->cpu];
+		grp = starget_to_session_grp(scsi_target(sc->device));
+		cls_session = grp->session_map[queue_id];
+		printk(KERN_ERR "using %u %s %s\n", queue_id,
+			dev_name(&grp->dev), dev_name(&cls_session->dev));
+	} else {
+		cls_session = starget_to_session(scsi_target(sc->device));
+	}
+
+	return cls_session->dd_data;
+}
+EXPORT_SYMBOL_GPL(scsi_cmd_to_session);
+
 enum {
 	FAILURE_BAD_HOST = 1,
 	FAILURE_SESSION_FAILED,
 <at>  <at>  -1643,7 +1669,6  <at>  <at>  enum {
 
 int iscsi_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *sc)
 {
-	struct iscsi_cls_session *cls_session;
 	struct iscsi_host *ihost;
 	int reason = 0;
 	struct iscsi_session *session;
 <at>  <at>  -1654,12 +1679,11  <at>  <at>  int iscsi_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *sc)
 	sc->SCp.ptr = NULL;
 
 	ihost = shost_priv(host);
+	session = scsi_cmd_to_session(sc);
 
-	cls_session = starget_to_session(scsi_target(sc->device));
-	session = cls_session->dd_data;
 	spin_lock_bh(&session->frwd_lock);
 
-	reason = iscsi_session_chkready(cls_session);
+	reason = iscsi_session_chkready(session->cls_session);
 	if (reason) {
 		sc->result = reason;
 		goto fault;
 <at>  <at>  -2120,16 +2144,13  <at>  <at>  static void iscsi_prep_abort_task_pdu(struct iscsi_task *task,
 
 int iscsi_eh_abort(struct scsi_cmnd *sc)
 {
-	struct iscsi_cls_session *cls_session;
 	struct iscsi_session *session;
 	struct iscsi_conn *conn;
 	struct iscsi_task *task;
 	struct iscsi_tm *hdr;
 	int rc, age;
 
-	cls_session = starget_to_session(scsi_target(sc->device));
-	session = cls_session->dd_data;
-
+	session = scsi_cmd_to_session(sc);
 	ISCSI_DBG_EH(session, "aborting sc %p\n", sc);
 
 	mutex_lock(&session->eh_mutex);
 <at>  <at>  -2260,15 +2281,12  <at>  <at>  static void iscsi_prep_lun_reset_pdu(struct scsi_cmnd *sc, struct iscsi_tm *hdr)
 
 int iscsi_eh_device_reset(struct scsi_cmnd *sc)
 {
-	struct iscsi_cls_session *cls_session;
 	struct iscsi_session *session;
 	struct iscsi_conn *conn;
 	struct iscsi_tm *hdr;
 	int rc = FAILED;
 
-	cls_session = starget_to_session(scsi_target(sc->device));
-	session = cls_session->dd_data;
-
+	session = scsi_cmd_to_session(sc);
 	ISCSI_DBG_EH(session, "LU Reset [sc %p lun %llu]\n", sc,
 		     sc->device->lun);
 
 <at>  <at>  -2355,12 +2373,10  <at>  <at>  EXPORT_SYMBOL_GPL(iscsi_session_recovery_timedout);
  */
 int iscsi_eh_session_reset(struct scsi_cmnd *sc)
 {
-	struct iscsi_cls_session *cls_session;
 	struct iscsi_session *session;
 	struct iscsi_conn *conn;
 
-	cls_session = starget_to_session(scsi_target(sc->device));
-	session = cls_session->dd_data;
+	session = scsi_cmd_to_session(sc);
 	conn = session->leadconn;
 
 	mutex_lock(&session->eh_mutex);
 <at>  <at>  -2423,15 +2439,12  <at>  <at>  static void iscsi_prep_tgt_reset_pdu(struct scsi_cmnd *sc, struct iscsi_tm *hdr)
  */
 int iscsi_eh_target_reset(struct scsi_cmnd *sc)
 {
-	struct iscsi_cls_session *cls_session;
 	struct iscsi_session *session;
 	struct iscsi_conn *conn;
 	struct iscsi_tm *hdr;
 	int rc = FAILED;
 
-	cls_session = starget_to_session(scsi_target(sc->device));
-	session = cls_session->dd_data;
-
+	session = scsi_cmd_to_session(sc);
 	ISCSI_DBG_EH(session, "tgt Reset [sc %p tgt %s]\n", sc,
 		     session->targetname);
 
 <at>  <at>  -2701,9 +2714,12  <at>  <at>  static void iscsi_host_dec_session_cnt(struct Scsi_Host *shost)
  * iscsi_session_setup - create iscsi cls session and host and session
  *  <at> iscsit: iscsi transport template
  *  <at> shost: scsi host
+ *  <at> grp: session group
  *  <at> cmds_max: session can queue
  *  <at> cmd_task_size: LLD task private data size
  *  <at> initial_cmdsn: initial CmdSN
+ *  <at> queue_id: mq queue id
+ *  <at> id: optional scsi target id
  *
  * This can be used by software iscsi_transports that allocate
  * a session per scsi host.
 <at>  <at>  -2714,8 +2730,9  <at>  <at>  static void iscsi_host_dec_session_cnt(struct Scsi_Host *shost)
  */
 struct iscsi_cls_session *
 iscsi_session_setup(struct iscsi_transport *iscsit, struct Scsi_Host *shost,
-		    uint16_t cmds_max, int dd_size, int cmd_task_size,
-		    uint32_t initial_cmdsn, unsigned int id)
+		    struct iscsi_session_grp *grp, uint16_t cmds_max,
+		    int dd_size, int cmd_task_size, uint32_t initial_cmdsn,
+		    uint32_t queue_id, unsigned int id)
 {
 	struct iscsi_host *ihost = shost_priv(shost);
 	struct iscsi_session *session;
 <at>  <at>  -2763,11 +2780,13  <at>  <at>  iscsi_session_setup(struct iscsi_transport *iscsit, struct Scsi_Host *shost,
 	}
 	scsi_cmds = total_cmds - ISCSI_MGMT_CMDS_MAX;
 
-	cls_session = iscsi_alloc_session(shost, iscsit,
+	cls_session = iscsi_alloc_session(iscsit, shost, grp,
 					  sizeof(struct iscsi_session) +
 					  dd_size);
 	if (!cls_session)
 		goto dec_session_count;
+	cls_session->queue_id = queue_id;
+
 	session = cls_session->dd_data;
 	session->cls_session = cls_session;
 	session->host = shost;
 <at>  <at>  -2810,8 +2829,11  <at>  <at>  iscsi_session_setup(struct iscsi_transport *iscsit, struct Scsi_Host *shost,
 		goto module_get_fail;
 
 	if (iscsi_add_session(cls_session, id))
+{
+printk(KERN_ERR "add session fail\n");
 		goto cls_session_fail;
 
+}
 	return cls_session;
 
 cls_session_fail:
diff --git a/drivers/scsi/qla4xxx/ql4_os.c b/drivers/scsi/qla4xxx/ql4_os.c
index 6d25879..2589dae 100644
--- a/drivers/scsi/qla4xxx/ql4_os.c
+++ b/drivers/scsi/qla4xxx/ql4_os.c
 <at>  <at>  -120,21 +120,24  <at>  <at>  static int qla4xxx_get_iface_param(struct iscsi_iface *iface,
 static enum blk_eh_timer_return qla4xxx_eh_cmd_timed_out(struct scsi_cmnd *sc);
 static struct iscsi_endpoint *qla4xxx_ep_connect(struct Scsi_Host *shost,
 						 struct sockaddr *dst_addr,
-						 int non_blocking);
+						 int non_blocking,
+						 uint32_t queue_id);
 static int qla4xxx_ep_poll(struct iscsi_endpoint *ep, int timeout_ms);
 static void qla4xxx_ep_disconnect(struct iscsi_endpoint *ep);
 static int qla4xxx_get_ep_param(struct iscsi_endpoint *ep,
 				enum iscsi_param param, char *buf);
 static int qla4xxx_conn_start(struct iscsi_cls_conn *conn);
 static struct iscsi_cls_conn *
-qla4xxx_conn_create(struct iscsi_cls_session *cls_sess, uint32_t conn_idx);
+qla4xxx_conn_create(struct iscsi_cls_session *cls_sess, uint32_t conn_idx,
+		    uint32_t queue_idx);
 static int qla4xxx_conn_bind(struct iscsi_cls_session *cls_session,
 			     struct iscsi_cls_conn *cls_conn,
 			     uint64_t transport_fd, int is_leading);
 static void qla4xxx_conn_destroy(struct iscsi_cls_conn *conn);
 static struct iscsi_cls_session *
-qla4xxx_session_create(struct iscsi_endpoint *ep, uint16_t cmds_max,
-			uint16_t qdepth, uint32_t initial_cmdsn);
+qla4xxx_session_create(struct iscsi_session_grp *grp, struct iscsi_endpoint *ep,
+			uint16_t cmds_max, uint16_t qdepth,
+			uint32_t initial_cmdsn, uint32_t queue_id);
 static void qla4xxx_session_destroy(struct iscsi_cls_session *sess);
 static void qla4xxx_task_work(struct work_struct *wdata);
 static int qla4xxx_alloc_pdu(struct iscsi_task *, uint8_t);
 <at>  <at>  -1658,7 +1661,7  <at>  <at>  static int qla4xxx_get_iface_param(struct iscsi_iface *iface,
 
 static struct iscsi_endpoint *
 qla4xxx_ep_connect(struct Scsi_Host *shost, struct sockaddr *dst_addr,
-		   int non_blocking)
+		   int non_blocking, uint32_t queue_id)
 {
 	int ret;
 	struct iscsi_endpoint *ep;
 <at>  <at>  -1835,8 +1838,8  <at>  <at>  static enum blk_eh_timer_return qla4xxx_eh_cmd_timed_out(struct scsi_cmnd *sc)
 	unsigned long flags;
 	enum blk_eh_timer_return ret = BLK_EH_NOT_HANDLED;
 
-	session = starget_to_session(scsi_target(sc->device));
-	sess = session->dd_data;
+	sess = scsi_cmd_to_session(sc);
+	session = sess->cls_session;
 
 	spin_lock_irqsave(&session->lock, flags);
 	if (session->state == ISCSI_SESSION_FAILED)
 <at>  <at>  -3036,9 +3039,9  <at>  <at>  static int qla4xxx_match_fwdb_session(struct scsi_qla_host *ha,
 }
 
 static struct iscsi_cls_session *
-qla4xxx_session_create(struct iscsi_endpoint *ep,
+qla4xxx_session_create(struct iscsi_session_grp *grp, struct iscsi_endpoint *ep,
 			uint16_t cmds_max, uint16_t qdepth,
-			uint32_t initial_cmdsn)
+			uint32_t initial_cmdsn, uint32_t queue_id)
 {
 	struct iscsi_cls_session *cls_sess;
 	struct scsi_qla_host *ha;
 <at>  <at>  -3065,9 +3068,9  <at>  <at>  qla4xxx_session_create(struct iscsi_endpoint *ep,
 		return NULL;
 
 	cls_sess = iscsi_session_setup(&qla4xxx_iscsi_transport, qla_ep->host,
-				       cmds_max, sizeof(struct ddb_entry),
+				       grp, cmds_max, sizeof(struct ddb_entry),
 				       sizeof(struct ql4_task_data),
-				       initial_cmdsn, ddb_index);
+				       initial_cmdsn, queue_id, ddb_index);
 	if (!cls_sess)
 		return NULL;
 
 <at>  <at>  -3144,7 +3147,8  <at>  <at>  destroy_session:
 }
 
 static struct iscsi_cls_conn *
-qla4xxx_conn_create(struct iscsi_cls_session *cls_sess, uint32_t conn_idx)
+qla4xxx_conn_create(struct iscsi_cls_session *cls_sess, uint32_t conn_idx,
+		    uint32_t queue_idx)
 {
 	struct iscsi_cls_conn *cls_conn;
 	struct iscsi_session *sess;
 <at>  <at>  -6574,7 +6578,7  <at>  <at>  static struct iscsi_endpoint *qla4xxx_get_ep_fwdb(struct scsi_qla_host *ha,
 		addr->sin_port = htons(le16_to_cpu(fw_ddb_entry->port));
 	}
 
-	ep = qla4xxx_ep_connect(ha->host, (struct sockaddr *)dst_addr, 0);
+	ep = qla4xxx_ep_connect(ha->host, (struct sockaddr *)dst_addr, 0, 0);
 	vfree(dst_addr);
 	return ep;
 }
 <at>  <at>  -6885,9 +6889,9  <at>  <at>  static int qla4xxx_sess_conn_setup(struct scsi_qla_host *ha,
 	 * the targer_id would get set when we issue the login
 	 */
 	cls_sess = iscsi_session_setup(&qla4xxx_iscsi_transport, ha->host,
-				       cmds_max, sizeof(struct ddb_entry),
+				       NULL, cmds_max, sizeof(struct ddb_entry),
 				       sizeof(struct ql4_task_data),
-				       initial_cmdsn, INVALID_ENTRY);
+				       initial_cmdsn, 0, INVALID_ENTRY);
 	if (!cls_sess) {
 		ret = QLA_ERROR;
 		goto exit_setup;
diff --git a/drivers/scsi/scsi_transport_iscsi.c b/drivers/scsi/scsi_transport_iscsi.c
index 67d43e3..18464720 100644
--- a/drivers/scsi/scsi_transport_iscsi.c
+++ b/drivers/scsi/scsi_transport_iscsi.c
 <at>  <at>  -79,6 +79,7  <at>  <at>  struct iscsi_internal {
 	struct transport_container session_cont;
 };
 
+static atomic_t iscsi_session_grp_nr; /* sysfs id for next new session group */
 static atomic_t iscsi_session_nr; /* sysfs session id for next new session */
 static struct workqueue_struct *iscsi_eh_timer_workq;
 
 <at>  <at>  -1728,8 +1729,10  <at>  <at>  static void iscsi_session_release(struct device *dev)
 	struct iscsi_cls_session *session = iscsi_dev_to_session(dev);
 	struct Scsi_Host *shost;
 
-	shost = iscsi_session_to_shost(session);
-	scsi_host_put(shost);
+	if (!session->grp) {
+		shost = dev_to_shost(dev);
+		scsi_host_put(shost);
+	}
 	ISCSI_DBG_TRANS_SESSION(session, "Completing session release\n");
 	kfree(session);
 }
 <at>  <at>  -1758,6 +1761,145  <at>  <at>  void iscsi_host_for_each_session(struct Scsi_Host *shost,
 }
 EXPORT_SYMBOL_GPL(iscsi_host_for_each_session);
 
+/*
+ * Group of sessions that can be used for multiqueue
+ */
+struct bus_type iscsi_session_grp_bus;
+
+static int iscsi_session_grp_bus_match(struct device *dev,
+				       struct device_driver *drv)
+{
+	if (dev->bus == &iscsi_session_grp_bus)
+		return 1;
+	return 0;
+}
+
+static void iscsi_session_grp_release(struct device *dev)
+{
+	struct iscsi_session_grp *grp = iscsi_dev_to_session_grp(dev);
+	struct device *parent = grp->dev.parent;
+
+	kfree(grp->session_map);
+	kfree(grp);
+	put_device(parent);
+}
+
+static int iscsi_is_session_grp_dev(const struct device *dev)
+{
+	return dev->type && dev->type->release == iscsi_session_grp_release;
+}
+
+struct device_type iscsi_session_grp_dev_type = {
+	.name		= "iscsi_session_grp_dev_type",
+	.release	= iscsi_session_grp_release,
+};
+
+struct bus_type iscsi_session_grp_bus = {
+	.name		= "iscsi_session_grp",
+	.match		= iscsi_session_grp_bus_match,
+};
+
+struct iscsi_session_grp *
+iscsi_create_session_grp(struct Scsi_Host *shost)
+{
+	struct iscsi_session_grp *grp;
+	int err;
+
+	grp = kzalloc(sizeof(*grp), GFP_KERNEL);
+	if (!grp)
+		return NULL;
+
+	grp->session_map = kzalloc(nr_cpu_ids *
+				   sizeof(struct iscsi_session_grp *),
+				   GFP_KERNEL);
+	if (!grp->session_map)
+		goto free_grp;
+
+	grp->max_sessions = nr_cpu_ids;
+	grp->target_id = ISCSI_MAX_TARGET;
+	grp->gid = atomic_add_return(1, &iscsi_session_grp_nr);
+	grp->dev.type = &iscsi_session_grp_dev_type;
+	grp->dev.bus = &iscsi_session_grp_bus;
+	/* released in grp release */
+	grp->dev.parent = get_device(&shost->shost_gendev);
+	dev_set_name(&grp->dev, "session_group-%u:%u",
+		     shost->host_no, grp->gid);
+	err = device_register(&grp->dev);
+	if (err) {
+		shost_printk(KERN_ERR, shost,
+			     "Could not create session group %s. Error %d\n",
+			     dev_name(&grp->dev), err);
+		goto free_map;
+	}
+	return grp;
+
+free_map:
+	kfree(grp->session_map);
+free_grp:
+	kfree(grp);
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(iscsi_create_session_grp);
+
+void iscsi_destroy_session_grp(struct iscsi_session_grp *grp)
+{
+	if (grp->session_count)
+		return;
+
+	device_unregister(&grp->dev);
+}
+EXPORT_SYMBOL_GPL(iscsi_destroy_session_grp);
+
+static int iscsi_session_grp_match_id(struct device *dev, void *data)
+{
+        struct iscsi_session_grp *grp;
+
+        if (!iscsi_session_grp_bus_match(dev, NULL))
+                return 0;
+
+        grp = iscsi_dev_to_session_grp(dev);
+        return (grp->gid == *((int *)data)) ? 1 : 0;
+}
+
+static struct iscsi_session_grp *
+iscsi_find_session_grp_by_ids(uint32_t host_no, uint32_t gid)
+{
+        struct iscsi_session_grp *grp = NULL;
+	struct Scsi_Host *shost;
+        struct device *dev;
+
+	shost = scsi_host_lookup(host_no);
+	if (!shost)
+		return NULL;
+
+        dev = device_find_child(&shost->shost_gendev, &gid,
+				iscsi_session_grp_match_id);
+        if (dev)
+                grp = iscsi_dev_to_session_grp(dev);
+
+	scsi_host_put(shost);
+        return grp;
+}
+
+static int iscsi_find_first_session(struct device *dev, void *data)
+{
+	return iscsi_is_session_dev(dev);
+}
+
+struct iscsi_cls_session *iscsi_dev_to_lead_session(struct device *dev)
+{
+	if (iscsi_is_session_dev(dev))
+		return iscsi_dev_to_session(dev);
+
+	/* dev is grp, so search it for first one we find */
+        dev = device_find_child(dev, NULL, iscsi_find_first_session);
+        if (dev)
+                return iscsi_dev_to_session(dev);
+	else
+		return NULL;
+}
+EXPORT_SYMBOL_GPL(iscsi_dev_to_lead_session);
+
 /**
  * iscsi_scan_finished - helper to report when running scans are done
  *  <at> shost: scsi host
 <at>  <at>  -1792,15 +1934,13  <at>  <at>  static int iscsi_user_scan_session(struct device *dev, void *data)
 	unsigned long flags;
 	unsigned int id;
 
-	if (!iscsi_is_session_dev(dev))
+	if (!iscsi_is_session_dev(dev) && !iscsi_is_session_grp_dev(dev))
 		return 0;
 
-	session = iscsi_dev_to_session(dev);
-
-	ISCSI_DBG_TRANS_SESSION(session, "Scanning session\n");
-
-	shost = iscsi_session_to_shost(session);
+	shost = dev_to_shost(dev);
 	ihost = shost->shost_data;
+	session = iscsi_dev_to_lead_session(dev);
+	ISCSI_DBG_TRANS_SESSION(session, "Scanning session\n");
 
 	mutex_lock(&ihost->mutex);
 	spin_lock_irqsave(&session->lock, flags);
 <at>  <at>  -1816,8 +1956,7  <at>  <at>  static int iscsi_user_scan_session(struct device *dev, void *data)
 		     scan_data->channel == 0) &&
 		    (scan_data->id == SCAN_WILD_CARD ||
 		     scan_data->id == id))
-			scsi_scan_target(&session->dev, 0, id,
-					 scan_data->lun, 1);
+			scsi_scan_target(dev, 0, id, scan_data->lun, 1);
 	}
 
 user_scan_exit:
 <at>  <at>  -1843,7 +1982,7  <at>  <at>  static void iscsi_scan_session(struct work_struct *work)
 {
 	struct iscsi_cls_session *session =
 			container_of(work, struct iscsi_cls_session, scan_work);
-	struct Scsi_Host *shost = iscsi_session_to_shost(session);
+	struct Scsi_Host *shost = dev_to_shost(&session->dev);
 	struct iscsi_cls_host *ihost = shost->shost_data;
 	struct iscsi_scan_data scan_data;
 
 <at>  <at>  -1923,7 +2062,7  <at>  <at>  static void __iscsi_unblock_session(struct work_struct *work)
 	struct iscsi_cls_session *session =
 			container_of(work, struct iscsi_cls_session,
 				     unblock_work);
-	struct Scsi_Host *shost = iscsi_session_to_shost(session);
+	struct Scsi_Host *shost = dev_to_shost(&session->dev);
 	struct iscsi_cls_host *ihost = shost->shost_data;
 	unsigned long flags;
 
 <at>  <at>  -1997,7 +2136,7  <at>  <at>  static void __iscsi_unbind_session(struct work_struct *work)
 	struct iscsi_cls_session *session =
 			container_of(work, struct iscsi_cls_session,
 				     unbind_work);
-	struct Scsi_Host *shost = iscsi_session_to_shost(session);
+	struct Scsi_Host *shost = dev_to_shost(&session->dev);
 	struct iscsi_cls_host *ihost = shost->shost_data;
 	unsigned long flags;
 	unsigned int target_id;
 <at>  <at>  -2026,9 +2165,43  <at>  <at>  static void __iscsi_unbind_session(struct work_struct *work)
 	ISCSI_DBG_TRANS_SESSION(session, "Completed target removal\n");
 }
 
+static int iscsi_session_grp_add_session(struct iscsi_cls_session *session)
+{
+	struct iscsi_session_grp *grp = session->grp;
+	int err;
+
+	if (!grp || session->queue_id >= grp->max_sessions)
+		return -EINVAL;
+
+	if (grp->session_map[session->queue_id]) {
+		struct iscsi_cls_session *existing;
+
+		existing = grp->session_map[session->queue_id];
+
+		iscsi_cls_session_printk(KERN_ERR, session,
+					 "%s already setup in map\n",
+					 dev_name(&existing->dev));
+		return -EINVAL;
+	}
+	grp->session_map[session->queue_id] = session;
+	grp->session_count++;
+
+	err = sysfs_create_link(&session->dev.kobj, &grp->dev.kobj,
+				"session_group");
+	if (err)
+		goto unmap;
+
+	return 0;
+
+unmap:
+	grp->session_map[session->queue_id] = NULL;
+	grp->session_count--;
+	return err;
+}
+
 struct iscsi_cls_session *
-iscsi_alloc_session(struct Scsi_Host *shost, struct iscsi_transport *transport,
-		    int dd_size)
+iscsi_alloc_session(struct iscsi_transport *transport, struct Scsi_Host *shost,
+		    struct iscsi_session_grp *grp, int dd_size)
 {
 	struct iscsi_cls_session *session;
 
 <at>  <at>  -2037,6 +2210,7  <at>  <at>  iscsi_alloc_session(struct Scsi_Host *shost, struct iscsi_transport *transport,
 	if (!session)
 		return NULL;
 
+	session->grp = grp;
 	session->transport = transport;
 	session->creator = -1;
 	session->recovery_tmo = 120;
 <at>  <at>  -2049,9 +2223,13  <at>  <at>  iscsi_alloc_session(struct Scsi_Host *shost, struct iscsi_transport *transport,
 	INIT_WORK(&session->scan_work, iscsi_scan_session);
 	spin_lock_init(&session->lock);
 
-	/* this is released in the dev's release function */
-	scsi_host_get(shost);
-	session->dev.parent = &shost->shost_gendev;
+	if (grp) {
+		session->dev.parent = &grp->dev;
+	} else {
+		/* this is released in the dev's release function */
+		scsi_host_get(shost);
+		session->dev.parent = &shost->shost_gendev;
+	}
 	session->dev.release = iscsi_session_release;
 	device_initialize(&session->dev);
 	if (dd_size)
 <at>  <at>  -2064,7 +2242,8  <at>  <at>  EXPORT_SYMBOL_GPL(iscsi_alloc_session);
 
 int iscsi_add_session(struct iscsi_cls_session *session, unsigned int target_id)
 {
-	struct Scsi_Host *shost = iscsi_session_to_shost(session);
+	struct Scsi_Host *shost = dev_to_shost(&session->dev);
+	struct iscsi_session_grp *grp = session->grp;
 	struct iscsi_cls_host *ihost;
 	unsigned long flags;
 	int id = 0;
 <at>  <at>  -2073,20 +2252,30  <at>  <at>  int iscsi_add_session(struct iscsi_cls_session *session, unsigned int target_id)
 	ihost = shost->shost_data;
 	session->sid = atomic_add_return(1, &iscsi_session_nr);
 
-	if (target_id == ISCSI_MAX_TARGET) {
+	if (grp && grp->target_id != ISCSI_MAX_TARGET) {
+		session->target_id = grp->target_id;
+	} else if (target_id == ISCSI_MAX_TARGET) {
 		id = ida_simple_get(&iscsi_sess_ida, 0, 0, GFP_KERNEL);
-
 		if (id < 0) {
 			iscsi_cls_session_printk(KERN_ERR, session,
 					"Failure in Target ID Allocation\n");
 			return id;
 		}
+
 		session->target_id = (unsigned int)id;
 		session->ida_used = true;
-	} else
+	} else {
 		session->target_id = target_id;
+	}
 
-	dev_set_name(&session->dev, "session%u", session->sid);
+	if (grp)
+		grp->target_id = session->target_id;
+//		dev_set_name(&session->dev, "session%u:%u:%u:%u",
+//			     shost->host_no, grp->gid, session->sid,
+//			     session->target_id);
+//	} else {
+		dev_set_name(&session->dev, "session%u", session->sid);
+//	}
 	err = device_add(&session->dev);
 	if (err) {
 		iscsi_cls_session_printk(KERN_ERR, session,
 <at>  <at>  -2099,10 +2288,18  <at>  <at>  int iscsi_add_session(struct iscsi_cls_session *session, unsigned int target_id)
 	list_add(&session->sess_list, &sesslist);
 	spin_unlock_irqrestore(&sesslock, flags);
 
+	if (grp) {
+		err = iscsi_session_grp_add_session(session);
+		if (err)
+			goto destroy_session;
+	}
+
 	iscsi_session_event(session, ISCSI_KEVENT_CREATE_SESSION);
 	ISCSI_DBG_TRANS_SESSION(session, "Completed session adding\n");
 	return 0;
 
+destroy_session:
+	iscsi_destroy_session(session);
 release_ida:
 	if (session->ida_used)
 		ida_simple_remove(&iscsi_sess_ida, session->target_id);
 <at>  <at>  -2126,7 +2323,7  <at>  <at>  iscsi_create_session(struct Scsi_Host *shost, struct iscsi_transport *transport,
 {
 	struct iscsi_cls_session *session;
 
-	session = iscsi_alloc_session(shost, transport, dd_size);
+	session = iscsi_alloc_session(transport, shost, NULL, dd_size);
 	if (!session)
 		return NULL;
 
 <at>  <at>  -2160,9 +2357,21  <at>  <at>  static int iscsi_iter_destroy_conn_fn(struct device *dev, void *data)
 	return iscsi_destroy_conn(iscsi_dev_to_conn(dev));
 }
 
+static void iscsi_session_grp_del_session(struct iscsi_cls_session *session)
+{
+	struct iscsi_session_grp *grp = session->grp;
+
+	if (!grp)
+		return;
+
+	grp->session_map[session->queue_id] = NULL;
+	grp->session_count--;
+	sysfs_remove_link(&session->dev.kobj, "session_group");
+}
+
 void iscsi_remove_session(struct iscsi_cls_session *session)
 {
-	struct Scsi_Host *shost = iscsi_session_to_shost(session);
+	struct Scsi_Host *shost = dev_to_shost(&session->dev);
 	unsigned long flags;
 	int err;
 
 <at>  <at>  -2201,6 +2410,7  <at>  <at>  void iscsi_remove_session(struct iscsi_cls_session *session)
 					 "for session. Error %d.\n", err);
 
 	transport_unregister_device(&session->dev);
+	iscsi_session_grp_del_session(session);
 
 	ISCSI_DBG_TRANS_SESSION(session, "Completing session removal\n");
 	device_del(&session->dev);
 <at>  <at>  -2649,7 +2859,7  <at>  <at>  int iscsi_session_event(struct iscsi_cls_session *session,
 	priv = iscsi_if_transport_lookup(session->transport);
 	if (!priv)
 		return -EINVAL;
-	shost = iscsi_session_to_shost(session);
+	shost = dev_to_shost(&session->dev);
 
 	skb = alloc_skb(len, GFP_KERNEL);
 	if (!skb) {
 <at>  <at>  -2702,22 +2912,62  <at>  <at>  int iscsi_session_event(struct iscsi_cls_session *session,
 EXPORT_SYMBOL_GPL(iscsi_session_event);
 
 static int
-iscsi_if_create_session(struct iscsi_internal *priv, struct iscsi_endpoint *ep,
-			struct iscsi_uevent *ev, pid_t pid,
-			uint32_t initial_cmdsn,	uint16_t cmds_max,
-			uint16_t queue_depth)
+iscsi_if_create_session(struct iscsi_internal *priv, struct iscsi_uevent *ev,
+			pid_t pid, int event_type)
 {
 	struct iscsi_transport *transport = priv->iscsi_transport;
 	struct iscsi_cls_session *session;
+	struct iscsi_endpoint *ep = NULL;
+	struct iscsi_session_grp *grp = NULL;
+	uint32_t initial_cmdsn;
+	uint16_t cmds_max;
+	uint16_t queue_depth;
 	struct Scsi_Host *shost;
+	uint32_t queue_id = 0;
+
+	switch (event_type) {
+	case ISCSI_UEVENT_CREATE_SESSION:
+		initial_cmdsn = ev->u.c_session.initial_cmdsn;
+		cmds_max = ev->u.c_session.cmds_max;
+		queue_depth = ev->u.c_session.queue_depth;
+		break;
+	case ISCSI_UEVENT_CREATE_BOUND_SESSION:
+		ep = iscsi_lookup_endpoint(ev->u.c_bound_session.ep_handle);
+		if (!ep)
+			return -EINVAL;
+
+		initial_cmdsn = ev->u.c_bound_session.initial_cmdsn;
+		cmds_max = ev->u.c_bound_session.cmds_max;
+		queue_depth = ev->u.c_bound_session.queue_depth;
+		break;
+	case ISCSI_UEVENT_MQ_CREATE_SESSION:
+		if (ev->u.c_mq_session.flags & ISCSI_UEVENT_FLAG_EP_BOUND) {
+			ep = iscsi_lookup_endpoint(ev->u.c_mq_session.ep_handle);
+			if (!ep)
+				return -EINVAL;
+		}
+		grp = iscsi_find_session_grp_by_ids(ev->u.c_mq_session.host_no,
+						    ev->u.c_mq_session.gid);
+		if (!grp) {
+			return -EINVAL;
+		}
+
+		queue_id = ev->u.c_mq_session.queue_id;
+		initial_cmdsn = ev->u.c_mq_session.initial_cmdsn;
+		cmds_max = ev->u.c_mq_session.cmds_max;
+		queue_depth = ev->u.c_mq_session.queue_depth;
+		break;
+	default:
+		return -ENOSYS;
+	}
 
-	session = transport->create_session(ep, cmds_max, queue_depth,
-					    initial_cmdsn);
+	session = transport->create_session(grp, ep, cmds_max, queue_depth,
+					    initial_cmdsn, queue_id);
 	if (!session)
 		return -ENOMEM;
 
 	session->creator = pid;
-	shost = iscsi_session_to_shost(session);
+	shost = dev_to_shost(&session->dev);
 	ev->r.c_session_ret.host_no = shost->host_no;
 	ev->r.c_session_ret.sid = session->sid;
 	ISCSI_DBG_TRANS_SESSION(session,
 <at>  <at>  -2726,19 +2976,31  <at>  <at>  iscsi_if_create_session(struct iscsi_internal *priv, struct iscsi_endpoint *ep,
 }
 
 static int
-iscsi_if_create_conn(struct iscsi_transport *transport, struct iscsi_uevent *ev)
+iscsi_if_create_conn(struct iscsi_transport *transport, struct iscsi_uevent *ev,
+		     int event_type)
 {
 	struct iscsi_cls_conn *conn;
 	struct iscsi_cls_session *session;
+	uint32_t queue_id = 0;
+	uint32_t sid, cid;
 
-	session = iscsi_session_lookup(ev->u.c_conn.sid);
+	if (event_type == ISCSI_UEVENT_MQ_CREATE_CONN) {
+		queue_id = ev->u.c_mq_conn.queue_id;
+		sid = ev->u.c_mq_conn.sid;
+		cid = ev->u.c_mq_conn.cid;
+	} else {
+		sid = ev->u.c_conn.sid;
+		cid = ev->u.c_conn.cid;
+	}
+
+	session = iscsi_session_lookup(sid);
 	if (!session) {
 		printk(KERN_ERR "iscsi: invalid session %d.\n",
 		       ev->u.c_conn.sid);
 		return -EINVAL;
 	}
 
-	conn = transport->create_conn(session, ev->u.c_conn.cid);
+	conn = transport->create_conn(session, cid, queue_id);
 	if (!conn) {
 		iscsi_cls_session_printk(KERN_ERR, session,
 					 "couldn't create a new connection.");
 <at>  <at>  -2801,11 +3063,13  <at>  <at>  static int iscsi_if_ep_connect(struct iscsi_transport *transport,
 	struct sockaddr *dst_addr;
 	struct Scsi_Host *shost = NULL;
 	int non_blocking, err = 0;
+	uint32_t queue_id = 0;
 
 	if (!transport->ep_connect)
 		return -EINVAL;
 
-	if (msg_type == ISCSI_UEVENT_TRANSPORT_EP_CONNECT_THROUGH_HOST) {
+	switch (msg_type) {
+	case ISCSI_UEVENT_TRANSPORT_EP_CONNECT_THROUGH_HOST:
 		shost = scsi_host_lookup(ev->u.ep_connect_through_host.host_no);
 		if (!shost) {
 			printk(KERN_ERR "ep connect failed. Could not find "
 <at>  <at>  -2814,11 +3078,29  <at>  <at>  static int iscsi_if_ep_connect(struct iscsi_transport *transport,
 			return -ENODEV;
 		}
 		non_blocking = ev->u.ep_connect_through_host.non_blocking;
-	} else
+		break;
+	case ISCSI_UEVENT_TRANSPORT_EP_CONNECT:
 		non_blocking = ev->u.ep_connect.non_blocking;
+		break;
+	case ISCSI_UEVENT_MQ_TRANSPORT_EP_CONNECT:
+		if (ev->u.ep_mq_connect.flags & ISCSI_UEVENT_FLAG_HOST_BOUND) {
+			shost = scsi_host_lookup(ev->u.ep_mq_connect.host_no);
+			if (!shost) {
+				printk(KERN_ERR "ep connect failed. Could not find host no %u\n",
+				       ev->u.ep_mq_connect.host_no);
+				return -ENODEV;
+			}
+		}
+
+		non_blocking = ev->u.ep_mq_connect.non_blocking;
+		queue_id = ev->u.ep_mq_connect.queue_id;
+		break;
+	default:
+		return -ENOSYS;
+	}
 
 	dst_addr = (struct sockaddr *)((char*)ev + sizeof(*ev));
-	ep = transport->ep_connect(shost, dst_addr, non_blocking);
+	ep = transport->ep_connect(shost, dst_addr, non_blocking, queue_id);
 	if (IS_ERR(ep)) {
 		err = PTR_ERR(ep);
 		goto release_host;
 <at>  <at>  -2864,6 +3146,7  <at>  <at>  iscsi_if_transport_ep(struct iscsi_transport *transport,
 	switch (msg_type) {
 	case ISCSI_UEVENT_TRANSPORT_EP_CONNECT_THROUGH_HOST:
 	case ISCSI_UEVENT_TRANSPORT_EP_CONNECT:
+	case ISCSI_UEVENT_MQ_TRANSPORT_EP_CONNECT:
 		rc = iscsi_if_ep_connect(transport, ev, msg_type);
 		break;
 	case ISCSI_UEVENT_TRANSPORT_EP_POLL:
 <at>  <at>  -3495,6 +3778,8  <at>  <at>  iscsi_if_recv_msg(struct sk_buff *skb, struct nlmsghdr *nlh, uint32_t *group)
 	struct iscsi_internal *priv;
 	struct iscsi_cls_session *session;
 	struct iscsi_cls_conn *conn;
+	struct Scsi_Host *shost;
+	struct iscsi_session_grp *grp;
 	struct iscsi_endpoint *ep = NULL;
 
 	if (nlh->nlmsg_type == ISCSI_UEVENT_PATH_UPDATE)
 <at>  <at>  -3512,24 +3797,30  <at>  <at>  iscsi_if_recv_msg(struct sk_buff *skb, struct nlmsghdr *nlh, uint32_t *group)
 
 	switch (nlh->nlmsg_type) {
 	case ISCSI_UEVENT_CREATE_SESSION:
-		err = iscsi_if_create_session(priv, ep, ev,
-					      NETLINK_CB(skb).portid,
-					      ev->u.c_session.initial_cmdsn,
-					      ev->u.c_session.cmds_max,
-					      ev->u.c_session.queue_depth);
-		break;
 	case ISCSI_UEVENT_CREATE_BOUND_SESSION:
-		ep = iscsi_lookup_endpoint(ev->u.c_bound_session.ep_handle);
-		if (!ep) {
-			err = -EINVAL;
-			break;
+	case ISCSI_UEVENT_MQ_CREATE_SESSION:
+		err = iscsi_if_create_session(priv, ev, NETLINK_CB(skb).portid,
+					      nlh->nlmsg_type);
+		break;
+	case ISCSI_UEVENT_MQ_CREATE_SESSION_GRP:
+		grp = transport->create_session_grp(
+						ev->u.c_mq_session_grp.host_no);
+		if (!grp) {
+			err = -ENOMEM;
+		} else {
+			shost = dev_to_shost(&grp->dev);
+			ev->r.retcode = 0;
+			ev->r.c_mq_session_grp_ret.host_no = shost->host_no;
+			ev->r.c_mq_session_grp_ret.gid = grp->gid;
 		}
-
-		err = iscsi_if_create_session(priv, ep, ev,
-					NETLINK_CB(skb).portid,
-					ev->u.c_bound_session.initial_cmdsn,
-					ev->u.c_bound_session.cmds_max,
-					ev->u.c_bound_session.queue_depth);
+		break;
+	case ISCSI_UEVENT_MQ_DESTROY_SESSION_GRP:
+		grp = iscsi_find_session_grp_by_ids(
+						ev->u.d_mq_session_grp.host_no,
+						ev->u.d_mq_session_grp.gid);
+		if (!grp)
+			return -EINVAL;
+		transport->destroy_session_grp(grp);
 		break;
 	case ISCSI_UEVENT_DESTROY_SESSION:
 		session = iscsi_session_lookup(ev->u.d_session.sid);
 <at>  <at>  -3541,13 +3832,14  <at>  <at>  iscsi_if_recv_msg(struct sk_buff *skb, struct nlmsghdr *nlh, uint32_t *group)
 	case ISCSI_UEVENT_UNBIND_SESSION:
 		session = iscsi_session_lookup(ev->u.d_session.sid);
 		if (session)
-			scsi_queue_work(iscsi_session_to_shost(session),
+			scsi_queue_work(dev_to_shost(&session->dev),
 					&session->unbind_work);
 		else
 			err = -EINVAL;
 		break;
 	case ISCSI_UEVENT_CREATE_CONN:
-		err = iscsi_if_create_conn(transport, ev);
+	case ISCSI_UEVENT_MQ_CREATE_CONN:
+		err = iscsi_if_create_conn(transport, ev, nlh->nlmsg_type);
 		break;
 	case ISCSI_UEVENT_DESTROY_CONN:
 		err = iscsi_if_destroy_conn(transport, ev);
 <at>  <at>  -3616,6 +3908,7  <at>  <at>  iscsi_if_recv_msg(struct sk_buff *skb, struct nlmsghdr *nlh, uint32_t *group)
 	case ISCSI_UEVENT_TRANSPORT_EP_POLL:
 	case ISCSI_UEVENT_TRANSPORT_EP_DISCONNECT:
 	case ISCSI_UEVENT_TRANSPORT_EP_CONNECT_THROUGH_HOST:
+	case ISCSI_UEVENT_MQ_TRANSPORT_EP_CONNECT:
 		err = iscsi_if_transport_ep(transport, ev, nlh->nlmsg_type);
 		break;
 	case ISCSI_UEVENT_TGT_DSCVR:
 <at>  <at>  -4022,6 +4315,17  <at>  <at>  show_priv_session_target_id(struct device *dev, struct device_attribute *attr,
 static ISCSI_CLASS_ATTR(priv_sess, target_id, S_IRUGO,
 			show_priv_session_target_id, NULL);
 
+static ssize_t
+show_priv_session_queue_id(struct device *dev, struct device_attribute *attr,
+			   char *buf)
+{
+	struct iscsi_cls_session *session = iscsi_dev_to_session(dev->parent);
+	return sprintf(buf, "%d\n", session->queue_id);
+}
+static ISCSI_CLASS_ATTR(priv_sess, queue_id, S_IRUGO,
+			show_priv_session_queue_id, NULL);
+
+
 #define iscsi_priv_session_attr_show(field, format)			\
 static ssize_t								\
 show_priv_session_##field(struct device *dev, 				\
 <at>  <at>  -4096,6 +4400,7  <at>  <at>  static struct attribute *iscsi_session_attrs[] = {
 	&dev_attr_priv_sess_creator.attr,
 	&dev_attr_sess_chap_out_idx.attr,
 	&dev_attr_sess_chap_in_idx.attr,
+	&dev_attr_priv_sess_queue_id.attr,
 	&dev_attr_priv_sess_target_id.attr,
 	&dev_attr_sess_auto_snd_tgt_disable.attr,
 	&dev_attr_sess_discovery_session.attr,
 <at>  <at>  -4210,6 +4515,8  <at>  <at>  static umode_t iscsi_session_attr_is_visible(struct kobject *kobj,
 		return S_IRUGO;
 	else if (attr == &dev_attr_priv_sess_target_id.attr)
 		return S_IRUGO;
+	else if (attr == &dev_attr_priv_sess_queue_id.attr)
+		return S_IRUGO;
 	else {
 		WARN_ONCE(1, "Invalid session attr");
 		return 0;
 <at>  <at>  -4349,15 +4656,13  <at>  <at>  EXPORT_SYMBOL_GPL(iscsi_get_port_state_name);
 static int iscsi_session_match(struct attribute_container *cont,
 			   struct device *dev)
 {
-	struct iscsi_cls_session *session;
 	struct Scsi_Host *shost;
 	struct iscsi_internal *priv;
 
 	if (!iscsi_is_session_dev(dev))
 		return 0;
 
-	session = iscsi_dev_to_session(dev);
-	shost = iscsi_session_to_shost(session);
+	shost = dev_to_shost(dev);
 	if (!shost->transportt)
 		return 0;
 
 <at>  <at>  -4371,17 +4676,13  <at>  <at>  static int iscsi_session_match(struct attribute_container *cont,
 static int iscsi_conn_match(struct attribute_container *cont,
 			   struct device *dev)
 {
-	struct iscsi_cls_session *session;
-	struct iscsi_cls_conn *conn;
 	struct Scsi_Host *shost;
 	struct iscsi_internal *priv;
 
 	if (!iscsi_is_conn_dev(dev))
 		return 0;
 
-	conn = iscsi_dev_to_conn(dev);
-	session = iscsi_dev_to_session(conn->dev.parent);
-	shost = iscsi_session_to_shost(session);
+	shost = dev_to_shost(dev);
 
 	if (!shost->transportt)
 		return 0;
 <at>  <at>  -4515,6 +4816,7  <at>  <at>  static __init int iscsi_transport_init(void)
 	printk(KERN_INFO "Loading iSCSI transport class v%s.\n",
 		ISCSI_TRANSPORT_VERSION);
 
+	atomic_set(&iscsi_session_grp_nr, 0);
 	atomic_set(&iscsi_session_nr, 0);
 
 	err = class_register(&iscsi_transport_class);
 <at>  <at>  -4545,10 +4847,14  <at>  <at>  static __init int iscsi_transport_init(void)
 	if (err)
 		goto unregister_session_class;
 
+	err = bus_register(&iscsi_session_grp_bus);
+	if (err)
+		goto unregister_flashnode_bus;
+
 	nls = netlink_kernel_create(&init_net, NETLINK_ISCSI, &cfg);
 	if (!nls) {
 		err = -ENOBUFS;
-		goto unregister_flashnode_bus;
+		goto unregister_session_grp_bus;
 	}
 
 	iscsi_eh_timer_workq = create_singlethread_workqueue("iscsi_eh");
 <at>  <at>  -4561,6 +4867,8  <at>  <at>  static __init int iscsi_transport_init(void)
 
 release_nls:
 	netlink_kernel_release(nls);
+unregister_session_grp_bus:
+	bus_unregister(&iscsi_session_grp_bus);
 unregister_flashnode_bus:
 	bus_unregister(&iscsi_flashnode_bus);
 unregister_session_class:
 <at>  <at>  -4582,6 +4890,7  <at>  <at>  static void __exit iscsi_transport_exit(void)
 {
 	destroy_workqueue(iscsi_eh_timer_workq);
 	netlink_kernel_release(nls);
+	bus_unregister(&iscsi_session_grp_bus);
 	bus_unregister(&iscsi_flashnode_bus);
 	transport_class_unregister(&iscsi_connection_class);
 	transport_class_unregister(&iscsi_session_class);
diff --git a/include/scsi/iscsi_if.h b/include/scsi/iscsi_if.h
index 95ed942..3faf644 100644
--- a/include/scsi/iscsi_if.h
+++ b/include/scsi/iscsi_if.h
 <at>  <at>  -72,6 +72,12  <at>  <at>  enum iscsi_uevent_e {
 	ISCSI_UEVENT_SET_CHAP		= UEVENT_BASE + 31,
 	ISCSI_UEVENT_GET_HOST_STATS	= UEVENT_BASE + 32,
 
+	ISCSI_UEVENT_MQ_CREATE_SESSION		= UEVENT_BASE + 33,
+	ISCSI_UEVENT_MQ_CREATE_SESSION_GRP	= UEVENT_BASE + 34,
+	ISCSI_UEVENT_MQ_DESTROY_SESSION_GRP	= UEVENT_BASE + 35,
+	ISCSI_UEVENT_MQ_CREATE_CONN		= UEVENT_BASE + 36,
+	ISCSI_UEVENT_MQ_TRANSPORT_EP_CONNECT	= UEVENT_BASE + 37,
+
 	/* up events */
 	ISCSI_KEVENT_RECV_PDU		= KEVENT_BASE + 1,
 	ISCSI_KEVENT_CONN_ERROR		= KEVENT_BASE + 2,
 <at>  <at>  -100,10 +106,13  <at>  <at>  enum iscsi_host_event_code {
 	ISCSI_EVENT_MAX,
 };
 
+#define ISCSI_UEVENT_FLAG_EP_BOUND	0x1
+#define ISCSI_UEVENT_FLAG_HOST_BOUND	0x1
+
 struct iscsi_uevent {
 	uint32_t type; /* k/u events type */
 	uint32_t iferror; /* carries interface or resource errors */
-	uint64_t transport_handle;
+	__aligned_u64 transport_handle;
 
 	union {
 		/* messages u -> k */
 <at>  <at>  -113,11 +122,21  <at>  <at>  struct iscsi_uevent {
 			uint16_t	queue_depth;
 		} c_session;
 		struct msg_create_bound_session {
-			uint64_t	ep_handle;
+			__aligned_u64	ep_handle;
 			uint32_t	initial_cmdsn;
 			uint16_t	cmds_max;
 			uint16_t	queue_depth;
 		} c_bound_session;
+		struct msg_mq_create_session {
+			__aligned_u64	ep_handle;
+			uint32_t	flags;
+			uint32_t	queue_id;
+			uint32_t	initial_cmdsn;
+			uint32_t	gid;
+			uint32_t	host_no;
+			uint16_t	cmds_max;
+			uint16_t	queue_depth;
+		} c_mq_session;
 		struct msg_destroy_session {
 			uint32_t	sid;
 		} d_session;
 <at>  <at>  -125,10 +144,15  <at>  <at>  struct iscsi_uevent {
 			uint32_t	sid;
 			uint32_t	cid;
 		} c_conn;
+		struct msg_mq_create_conn {
+			uint32_t	sid;
+			uint32_t	cid;
+			uint32_t	queue_id;
+		} c_mq_conn;
 		struct msg_bind_conn {
 			uint32_t	sid;
 			uint32_t	cid;
-			uint64_t	transport_eph;
+			__aligned_u64	transport_eph;
 			uint32_t	is_leading;
 		} b_conn;
 		struct msg_destroy_conn {
 <at>  <at>  -154,7 +178,7  <at>  <at>  struct iscsi_uevent {
 		struct msg_stop_conn {
 			uint32_t	sid;
 			uint32_t	cid;
-			uint64_t	conn_handle;
+			__aligned_u64	conn_handle;
 			uint32_t	flag;
 		} stop_conn;
 		struct msg_get_stats {
 <at>  <at>  -168,12 +192,21  <at>  <at>  struct iscsi_uevent {
 			uint32_t	host_no;
 			uint32_t	non_blocking;
 		} ep_connect_through_host;
+		struct msg_mq_transport_connect {
+			uint32_t	flags;
+			uint32_t	host_no;
+			uint32_t	non_blocking;
+			uint32_t	queue_id;
+			/*
+			 * TODO: Sagi/Or, there were some new fields we wanted for iser multipath right (from a past issue and not related to mq)? Let's add them here now while we are at it.
+			 */
+		} ep_mq_connect;
 		struct msg_transport_poll {
-			uint64_t	ep_handle;
+			__aligned_u64	ep_handle;
 			uint32_t	timeout_ms;
 		} ep_poll;
 		struct msg_transport_disconnect {
-			uint64_t	ep_handle;
+			__aligned_u64	ep_handle;
 		} ep_disconnect;
 		struct msg_tgt_dscvr {
 			enum iscsi_tgt_dscvr	type;
 <at>  <at>  -244,12 +277,23  <at>  <at>  struct iscsi_uevent {
 			uint32_t	sid;
 		} logout_flashnode_sid;
 		struct msg_get_host_stats {
-			uint32_t host_no;
+			uint32_t	host_no;
 		} get_host_stats;
+		struct msg_mq_create_session_grp {
+			uint32_t	host_no;
+		} c_mq_session_grp;
+		struct msg_mq_destroy_session_grp {
+			uint32_t	host_no;
+			uint32_t	gid;
+		} d_mq_session_grp;
 	} u;
 	union {
 		/* messages k -> u */
 		int			retcode;
+		struct msg_mq_create_session_grp_ret {
+			uint32_t	gid;
+			uint32_t	host_no;
+		} c_mq_session_grp_ret;
 		struct msg_create_session_ret {
 			uint32_t	sid;
 			uint32_t	host_no;
 <at>  <at>  -265,7 +309,7  <at>  <at>  struct iscsi_uevent {
 		struct msg_recv_req {
 			uint32_t	sid;
 			uint32_t	cid;
-			uint64_t	recv_handle;
+			__aligned_u64	recv_handle;
 		} recv_req;
 		struct msg_conn_login {
 			uint32_t        sid;
 <at>  <at>  -282,7 +326,7  <at>  <at>  struct iscsi_uevent {
 			uint32_t	sid;
 		} d_session;
 		struct msg_transport_connect_ret {
-			uint64_t	handle;
+			__aligned_u64	handle;
 		} ep_connect_ret;
 		struct msg_req_path {
 			uint32_t	host_no;
 <at>  <at>  -620,6 +664,7  <at>  <at>  enum iscsi_param {
 	ISCSI_PARAM_DISCOVERY_PARENT_IDX,
 	ISCSI_PARAM_DISCOVERY_PARENT_TYPE,
 	ISCSI_PARAM_LOCAL_IPADDR,
+
 	/* must always be last */
 	ISCSI_PARAM_MAX,
 };
diff --git a/include/scsi/libiscsi.h b/include/scsi/libiscsi.h
index 4d1c46a..74b289e2 100644
--- a/include/scsi/libiscsi.h
+++ b/include/scsi/libiscsi.h
 <at>  <at>  -408,7 +408,8  <at>  <at>  extern int iscsi_target_alloc(struct scsi_target *starget);
  */
 extern struct iscsi_cls_session *
 iscsi_session_setup(struct iscsi_transport *, struct Scsi_Host *shost,
-		    uint16_t, int, int, uint32_t, unsigned int);
+		    struct iscsi_session_grp *, uint16_t, int, int, uint32_t,
+		    unsigned int, uint32_t);
 extern void iscsi_session_teardown(struct iscsi_cls_session *);
 extern void iscsi_session_recovery_timedout(struct iscsi_cls_session *);
 extern int iscsi_set_param(struct iscsi_cls_conn *cls_conn,
 <at>  <at>  -473,6 +474,7  <at>  <at>  extern void iscsi_complete_scsi_task(struct iscsi_task *task,
 extern void iscsi_pool_free(struct iscsi_pool *);
 extern int iscsi_pool_init(struct iscsi_pool *, int, void ***, int);
 extern int iscsi_switch_str_param(char **, char *);
+extern struct iscsi_session *scsi_cmd_to_session(struct scsi_cmnd *sc);
 
 /*
  * inline functions to deal with padding.
diff --git a/include/scsi/scsi_transport_iscsi.h b/include/scsi/scsi_transport_iscsi.h
index 2555ee5..6d94384 100644
--- a/include/scsi/scsi_transport_iscsi.h
+++ b/include/scsi/scsi_transport_iscsi.h
 <at>  <at>  -41,12 +41,15  <at>  <at>  struct iscsi_iface;
 struct bsg_job;
 struct iscsi_bus_flash_session;
 struct iscsi_bus_flash_conn;
+struct iscsi_session_grp;
 
 /**
  * struct iscsi_transport - iSCSI Transport template
  *
  *  <at> name:		transport name
  *  <at> caps:		iSCSI Data-Path capabilities
+ *  <at> create_session_grp:	create session group object and host if needed
+ *  <at> destroy_session_grp: destroy group and host
  *  <at> create_session:	create new iSCSI session object
  *  <at> destroy_session:	destroy existing iSCSI session object
  *  <at> create_conn:	create new iSCSI connection
 <at>  <at>  -89,12 +92,17  <at>  <at>  struct iscsi_transport {
 	char *name;
 	unsigned int caps;
 
-	struct iscsi_cls_session *(*create_session) (struct iscsi_endpoint *ep,
+	struct iscsi_session_grp *(*create_session_grp) (uint32_t host_no);
+	void (*destroy_session_grp) (struct iscsi_session_grp *grp);
+	struct iscsi_cls_session *(*create_session) (
+					struct iscsi_session_grp *grp,
+					struct iscsi_endpoint *ep,
 					uint16_t cmds_max, uint16_t qdepth,
-					uint32_t sn);
+					uint32_t sn,
+					uint32_t queue_id);
 	void (*destroy_session) (struct iscsi_cls_session *session);
 	struct iscsi_cls_conn *(*create_conn) (struct iscsi_cls_session *sess,
-				uint32_t cid);
+				uint32_t cid, uint32_t queue_id);
 	int (*bind_conn) (struct iscsi_cls_session *session,
 			  struct iscsi_cls_conn *cls_conn,
 			  uint64_t transport_eph, int is_leading);
 <at>  <at>  -133,7 +141,8  <at>  <at>  struct iscsi_transport {
 	void (*session_recovery_timedout) (struct iscsi_cls_session *session);
 	struct iscsi_endpoint *(*ep_connect) (struct Scsi_Host *shost,
 					      struct sockaddr *dst_addr,
-					      int non_blocking);
+					      int non_blocking,
+					      uint32_t queue_id);
 	int (*ep_poll) (struct iscsi_endpoint *ep, int timeout_ms);
 	void (*ep_disconnect) (struct iscsi_endpoint *ep);
 	int (*tgt_dscvr) (struct Scsi_Host *shost, enum iscsi_tgt_dscvr type,
 <at>  <at>  -232,6 +241,7  <at>  <at>  enum {
 
 struct iscsi_cls_session {
 	struct list_head sess_list;		/* item in session_list */
+	struct list_head *grp_list;
 	struct iscsi_transport *transport;
 	spinlock_t lock;
 	struct work_struct block_work;
 <at>  <at>  -246,6 +256,8  <at>  <at>  struct iscsi_cls_session {
 	unsigned int target_id;
 	bool ida_used;
 
+	unsigned int queue_id;
+	struct iscsi_session_grp *grp;
 	/*
 	 * pid of userspace process that created session or -1 if
 	 * created by the kernel.
 <at>  <at>  -263,11 +275,25  <at>  <at>  struct iscsi_cls_session {
 #define transport_class_to_session(_cdev) \
 	iscsi_dev_to_session(_cdev->parent)
 
-#define iscsi_session_to_shost(_session) \
-	dev_to_shost(_session->dev.parent)
-
 #define starget_to_session(_stgt) \
-	iscsi_dev_to_session(_stgt->dev.parent)
+	iscsi_dev_to_lead_session(_stgt->dev.parent)
+
+struct iscsi_session_grp {
+	struct device dev;
+	unsigned int target_id;
+	uint32_t gid;
+	/* hctx idx to cls_session mapping */
+	struct iscsi_cls_session **session_map;
+	uint32_t session_count;
+	uint32_t max_sessions;
+	struct list_head *session_list;
+};
+
+#define iscsi_dev_to_session_grp(_dev) \
+	container_of(_dev, struct iscsi_session_grp, dev)
+
+#define starget_to_session_grp(_stgt) \
+	iscsi_dev_to_session_grp(_stgt->dev.parent)
 
 struct iscsi_cls_host {
 	atomic_t nr_scans;
 <at>  <at>  -419,10 +445,15  <at>  <at>  struct iscsi_bus_flash_session {
 #define iscsi_cls_conn_printk(prefix, _cls_conn, fmt, a...) \
 	dev_printk(prefix, &(_cls_conn)->dev, fmt, ##a)
 
+extern struct iscsi_cls_session *iscsi_dev_to_lead_session(struct device *dev);
+extern struct iscsi_session_grp *
+iscsi_create_session_grp(struct Scsi_Host *shost);
+extern void iscsi_destroy_session_grp(struct iscsi_session_grp *grp);
 extern int iscsi_session_chkready(struct iscsi_cls_session *session);
 extern int iscsi_is_session_online(struct iscsi_cls_session *session);
-extern struct iscsi_cls_session *iscsi_alloc_session(struct Scsi_Host *shost,
-				struct iscsi_transport *transport, int dd_size);
+extern struct iscsi_cls_session *
+iscsi_alloc_session(struct iscsi_transport *transport, struct Scsi_Host *shost,
+		    struct iscsi_session_grp *grp, int dd_size);
 extern int iscsi_add_session(struct iscsi_cls_session *session,
 			     unsigned int target_id);
 extern int iscsi_session_event(struct iscsi_cls_session *session,
Andy Grover | 7 Feb 01:36 2015
Picon

[PATCHv2 0/3] Fix issues resulting from actor rewrite

I think I got a handle on the original issue you were seeing Mike,
please take a look at #2 patch.

The added debug logging had some bugs, so v2 also respins that patch,
now #3.

Thanks -- Andy

Andy Grover (3):
  Fix incorrect list operation leading to out-of-order items on
    pend_list
  Prevent spinning over poll() when reconnecting to an inaccessible
    target
  Add some more debug logging to actor.c

 usr/actor.c     | 27 +++++++++++++++++++++------
 usr/initiator.c | 11 +++++------
 2 files changed, 26 insertions(+), 12 deletions(-)

-- 
2.1.0

--

-- 
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe@...
To post to this group, send email to open-iscsi@...
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Nicolas Ecarnot | 6 Feb 15:49 2015
Picon

2 NICs, 2 targets : discover OK but no login. Same subnet impossible?

Hello list,

My setup is described here in details :
https://groups.google.com/d/msg/open-iscsi/ivprFF9hklY/mkVPHo5q1p0J
but now I have another issue.

I'm trying to connect 2 NICs to a SAN that presents 2 ip, and all that is on the same subnet and on the same switch.
I read some old comments on old threads (< 2006) that discouraged that, and one answer of Mike also said the same :
I think you might hit some issues or at least not get what you expect with both network interfaces on the same subnet, but using the default iscsi default iface.
(though I don't understand the very last part after the comma)

But I hoped the situation had improved since, and that some limitations had gone.
Indeed, when re-re-re-reading the doc page of open-iscsi, I see nowhere advicing against having two NICs on the same subnet.

So I tried and got that :
- both NICs (em3 and em4) can ping the 2 SAN's ip
- With iscsiadm, I'm adding iface0 and iface1, and I specify the MAC hardware address of the corresponding em3 and em4
- When discovering, all seems to work well
- When login in, it is failing in a way it seems I'm not allowed to connect using both ifaces at the same time.

When trying to log in, I specify the iface0, and the login to the 2 SAN ip is working well.
But then, doing the same with iface1 is failing, with a kernel errror message :
iscsid: conn 0 login rejected: target error (03/02)
And after having logout, trying the same way around with iface1 first results in the same behaviour.

Is it by design that iscsi does not permit the login from 2 ifaces concurrently?
I tried to find somewhere in my EMC SAN (Cx3-10) where there would be some sort of such limitations, but found nothing.

--
Nicolas Ecarnot

--
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To post to this group, send email to open-iscsi-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
Andy Grover | 6 Feb 02:14 2015
Picon

[PATCH 0/2] Fix a bug in actor.c and add more debug stuff

Hi Mike,

Patch 1 fixes the timers to expire properly. Patch 2 adds some more debug
prints if you think that would be useful beyond the present issues.

I haven't tried digging into the original issue, but will be geting into it
tomorrow.

Andy Grover (2):
  Fix incorrect list operation leading to out-of-order items on
    pend_list
  Add some more debug logging to actor.c

 usr/actor.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

-- 
2.1.0

--

-- 
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe@...
To post to this group, send email to open-iscsi@...
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

nicolas.ecarnot | 4 Feb 14:28 2015
Picon

Logging hanging, invalid session

Hi list,

I'm facing a similar problem as what has been discussed here in 2011 :
https://groups.google.com/d/msg/open-iscsi/pQQ8_ernzgQ/7ZVkoDmY2qYJ
but versions have since raised, so I don't know what to look at.

TL;DR :
iscsiadm -m node -T iqn.1992-04.com.emc:cx.ckm00094900174.a2 -l
...
...
-> sleeping there until I ctrl-c interrupt it.
The only error msg I get is on the console :
iscsi: invalid session 0
Specifying the SAN ip does not improve anything.

Long version :

We have a Dell PE R610 in 64bits with 4 NICs :
- em1 + em2 are bonded and dedicated to the management on a specific VLAN
- em3 and em4 are not bonded, and dedicated to the iSCSI specific VLAN

ping -I from em3 and em4 towards our Cx4-120 SAN is working, towards its two different IPs.
And the ping from the SAN is also working.

In /etc/sysctl.conf, I have :
net.ipv4.conf.em3.arp_ignore = 1
net.ipv4.conf.em4.arp_ignore = 1
net.ipv4.conf.em3.arp_announce = 2
net.ipv4.conf.em4.arp_announce = 2
net.ipv4.conf.em3.rp_filter = 0
net.ipv4.conf.em4.rp_filter = 0


iptables and selinux are off.

LSB Version:    :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID:    OracleServer
Description:    Oracle Linux Server release 6.6


The discovery sounds good :
root <at> serv3:/var/lib/iscsi# iscsiadm -m discovery -p 192.168.29.1 -t st -I default
192.168.29.1:3260,1 iqn.1992-04.com.emc:cx.ckm00094900174.a2
192.168.29.2:3260,2 iqn.1992-04.com.emc:cx.ckm00094900174.b2


The 2 NICs are NetXtreme II BCM5709 Gigabit Ethernet. They support offloading, but I'm not comfortable enough with that yet to play with it. So at present, I'd prefer just use a classical tcp transport.

Iface scanning sounds good too :
root <at> serv3:/var/lib/iscsi# iscsiadm -m iface
default tcp,<empty>,<empty>,<empty>,<empty>
iser iser,<empty>,<empty>,<empty>,<empty>
bnx2i.f0:4d:a2:07:59:91 bnx2i,f0:4d:a2:07:59:91,<empty>,<empty>,<empty>
bnx2i.00:00:00:00:00:00 bnx2i,00:00:00:00:00:00,<empty>,<empty>,<empty>
bnx2i.f0:4d:a2:07:59:95 bnx2i,f0:4d:a2:07:59:95,<empty>,<empty>,<empty>
bnx2i.f0:4d:a2:07:59:93 bnx2i,f0:4d:a2:07:59:93,<empty>,<empty>,<empty>


Other informations :
root <at> serv3:/var/lib/iscsi# rpm -qa|grep -i iscsi
iscsi-initiator-utils-6.2.0.873-2.0.2.el6.i686
root <at> serv3:/var/lib/iscsi# iscsiuio -v
iscsiuio: Version '0.7.6.1', Build Date: 'Wed May 29 14:08:11 PDT 2013'
root <at> serv3:/var/lib/iscsi# uname -a
Linux serv3 3.8.13-55.1.5.el6uek.x86_64 #2 SMP Wed Jan 28 17:03:28 PST 2015 x86_64 x86_64 x86_64 GNU/Linux


I tried to disable em4, reboot, and try a connection from a single NIC (em3), but that did nothing better.

I don't know if the fact that these NICs support offloading may disturb the simplest tcp transport way of connecting?
I must admit I'm a little blocked here, as this is not the first server I'm connecting using iscsi, but as the pings are OK, what else could I check?

Thank you.

--
Nicolas Ecarnot

--
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To post to this group, send email to open-iscsi-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
Nicolas Ecarnot | 4 Feb 14:44 2015
Picon

Logging hanging. Invalid session

Hi list,

I'm facing a similar problem as what has been discussed here in 2011 :
https://groups.google.com/d/msg/open-iscsi/pQQ8_ernzgQ/7ZVkoDmY2qYJ
but versions have since raised, so I don't know what to look at.

TL;DR :
iscsiadm -m node -T iqn.1992-04.com.emc:cx.ckm00094900174.a2 -l
...
...
-> sleeping there until I ctrl-c interrupt it.
The only error msg I get is on the console :
iscsi: invalid session 0
Specifying the SAN ip does not improve anything.

Long version :

We have a Dell PE R610 in 64bits with 4 NICs :
- em1 + em2 are bonded and dedicated to the management on a specific VLAN
- em3 and em4 are not bonded, and dedicated to the iSCSI specific VLAN

ping -I from em3 and em4 towards our EMC Cx4-120 SAN is working, towards its two different IPs.
And the ping from the SAN is also working.

In /etc/sysctl.conf, I have :
net.ipv4.conf.em3.arp_ignore = 1
net.ipv4.conf.em4.arp_ignore = 1
net.ipv4.conf.em3.arp_announce = 2
net.ipv4.conf.em4.arp_announce = 2
net.ipv4.conf.em3.rp_filter = 0
net.ipv4.conf.em4.rp_filter = 0


iptables and selinux are off.

LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID:    OracleServer
Description:    Oracle Linux Server release 6.6


The discovery sounds good :
root <at> serv3:/var/lib/iscsi# iscsiadm -m discovery -p 192.168.29.1 -t st -I default
192.168.29.1:3260,1 iqn.1992-04.com.emc:cx.ckm00094900174.a2
192.168.29.2:3260,2 iqn.1992-04.com.emc:cx.ckm00094900174.b2


The 2 NICs are NetXtreme II BCM5709 Gigabit Ethernet. They support offloading, but I'm not comfortable enough with that yet to play with it. So at present, I'd prefer just use a classical tcp transport.

Iface scanning sounds good too :
root <at> serv3:/var/lib/iscsi# iscsiadm -m iface
default tcp,<empty>,<empty>,<empty>,<empty>
iser iser,<empty>,<empty>,<empty>,<empty>
bnx2i.f0:4d:a2:07:59:91 bnx2i,f0:4d:a2:07:59:91,<empty>,<empty>,<empty>
bnx2i.00:00:00:00:00:00 bnx2i,00:00:00:00:00:00,<empty>,<empty>,<empty>
bnx2i.f0:4d:a2:07:59:95 bnx2i,f0:4d:a2:07:59:95,<empty>,<empty>,<empty>
bnx2i.f0:4d:a2:07:59:93 bnx2i,f0:4d:a2:07:59:93,<empty>,<empty>,<empty>


Other informations :
root <at> serv3:/var/lib/iscsi# rpm -qa|grep -i iscsi
iscsi-initiator-utils-6.2.0.873-2.0.2.el6.i686
root <at> serv3:/var/lib/iscsi# iscsiuio -v
iscsiuio: Version '0.7.6.1', Build Date: 'Wed May 29 14:08:11 PDT 2013'
root <at> serv3:/var/lib/iscsi# uname -a
Linux serv3 3.8.13-55.1.5.el6uek.x86_64 #2 SMP Wed Jan 28 17:03:28 PST 2015 x86_64 x86_64 x86_64 GNU/Linux


I tried to disable em4, reboot, and try a connection from a single NIC (em3), but that did nothing better.

I don't know if the fact that these NICs support offloading may disturb the simplest tcp transport way of connecting?
I must admit I'm a little blocked here, as this is not the first server I'm connecting using iscsi, but as the pings are OK, what else could I check?

Thank you.

-- 
Nicolas Ecarnot

--
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To post to this group, send email to open-iscsi-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
Chris Leech | 4 Feb 01:28 2015
Picon

[PATCH] iscsid safe session logout

Implement a safe logout option, which uses libmount from util-linux to
check for active mounts (and swaps) over devices, their partitions, and
any holders (like LVM and multipath device maps).  When enabled iscsid
will refuse to logout of sessions actively being used for mounts,
returning a status of EBUSY to the ipc request.

I've made it a configuration option (iscsid.safe_logout) that defaults
to "No" to preserve the existing behavior as the default, while making
it available for users that prefer a safety check.

This does add a new dependency on libmount.

Signed-off-by: Chris Leech <cleech@...>
---
 etc/iscsid.conf |   3 +
 usr/Makefile    |   4 +-
 usr/initiator.c | 199 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 usr/sysfs.c     |  40 ++++++++++++
 usr/sysfs.h     |   4 ++
 5 files changed, 248 insertions(+), 2 deletions(-)

diff --git a/etc/iscsid.conf b/etc/iscsid.conf
index ef76dc0..6d9a5c0 100644
--- a/etc/iscsid.conf
+++ b/etc/iscsid.conf
 <at>  <at>  -22,6 +22,9  <at>  <at> 
 # Default for upstream open-iscsi scripts (uncomment to activate).
 iscsid.startup = /sbin/iscsid

+# Check for active mounts on devices reachable through a session
+# and refuse to logout if there are any.  Defaults to "No".
+# iscsid.safe_logout = Yes

 #############################
 # NIC/HBA and driver settings
diff --git a/usr/Makefile b/usr/Makefile
index 550fdff..9209d5d 100644
--- a/usr/Makefile
+++ b/usr/Makefile
 <at>  <at>  -55,14 +55,14  <at>  <at>  all: $(PROGRAMS)

 iscsid: $(ISCSI_LIB_SRCS) $(INITIATOR_SRCS) $(DISCOVERY_SRCS) \
 	iscsid.o session_mgmt.o discoveryd.o
-	$(CC) $(CFLAGS) $^ -o $ <at>   -L../utils/open-isns -lisns -lrt
+	$(CC) $(CFLAGS) $^ -o $ <at>   -L../utils/open-isns -lisns -lrt -lmount

 iscsiadm: $(ISCSI_LIB_SRCS) $(DISCOVERY_SRCS) iscsiadm.o session_mgmt.o
 	$(CC) $(CFLAGS) $^ -o $ <at>  -L../utils/open-isns -lisns

 iscsistart: $(ISCSI_LIB_SRCS) $(INITIATOR_SRCS) $(FW_BOOT_SRCS) \
 		iscsistart.o statics.o
-	$(CC) $(CFLAGS) -static $^ -o $ <at>  -lrt
+	$(CC) $(CFLAGS) $^ -o $ <at>  -lrt -lmount
 clean:
 	rm -f *.o $(PROGRAMS) .depend $(LIBSYS)

diff --git a/usr/initiator.c b/usr/initiator.c
index 1aadc9b..bedc4a8 100644
--- a/usr/initiator.c
+++ b/usr/initiator.c
 <at>  <at>  -30,6 +30,7  <at>  <at> 
 #include <errno.h>
 #include <dirent.h>
 #include <fcntl.h>
+#include <libmount/libmount.h>

 #include "initiator.h"
 #include "transport.h"
 <at>  <at>  -2140,11 +2141,200  <at>  <at>  static int session_unbind(struct iscsi_session *session)
 	return err;
 }

+static struct libmnt_table *mtab, *swaps;
+
+static void libmount_cleanup(void)
+{
+	mnt_free_table(mtab);
+	mnt_free_table(swaps);
+	mtab = swaps = NULL;
+}
+
+static int libmount_init(void)
+{
+	mnt_init_debug(0);
+	mtab = mnt_new_table();
+	swaps = mnt_new_table();
+	if (!mtab || !swaps) {
+		libmount_cleanup();
+		return -ENOMEM;
+	}
+	mnt_table_parse_mtab(mtab, NULL);
+	mnt_table_parse_swaps(swaps, NULL);
+	return 0;
+}
+
+static int trans_filter(const struct dirent *d)
+{
+	if (!strcmp(".", d->d_name) || !strcmp("..", d->d_name))
+		return 0;
+	return 1;
+}
+
+static int subdir_filter(const struct dirent *d)
+{
+	if (!(d->d_type & DT_DIR))
+		return 0;
+	return trans_filter(d);
+}
+
+static int is_partition(const char *path)
+{
+	char *devtype;
+	int rc = 0;
+
+	devtype = sysfs_get_uevent_devtype(path);
+	if (!devtype)
+		return 0;
+	if (strcmp(devtype, "partition") == 0)
+		rc = 1;
+	free(devtype);
+	return rc;
+}
+
+static int blockdev_check_mnts(char *syspath)
+{
+	struct libmnt_fs *fs;
+	char *devname = NULL;
+	char *_devname = NULL;
+	int rc = 0;
+
+	devname = sysfs_get_uevent_devname(syspath);
+	if (!devname)
+		goto out;
+
+	_devname = calloc(1, PATH_MAX);
+	if (!_devname)
+		goto out;
+	snprintf(_devname, PATH_MAX, "/dev/%s", devname);
+
+	fs = mnt_table_find_source(mtab, _devname, MNT_ITER_FORWARD);
+	if (fs) {
+		rc = 1;
+		goto out;
+	}
+	fs = mnt_table_find_source(swaps, _devname, MNT_ITER_FORWARD);
+	if (fs)
+		rc = 1;
+out:
+	free(devname);
+	free(_devname);
+	return rc;
+}
+
+static int count_device_users(char *syspath);
+
+static int blockdev_get_partitions(char *syspath)
+{
+	struct dirent **parts = NULL;
+	int n, i;
+	int count = 0;
+
+	n = scandir(syspath, &parts, subdir_filter, alphasort);
+	for (i = 0; i < n; i++) {
+		char *newpath;
+
+		newpath = calloc(1, PATH_MAX);
+		if (!newpath)
+			continue;
+		snprintf(newpath, PATH_MAX, "%s/%s", syspath, parts[i]->d_name);
+		free(parts[i]);
+		if (is_partition(newpath)) {
+			count += count_device_users(newpath);
+		}
+		free(newpath);
+	}
+	free(parts);
+	return count;
+}
+
+static int blockdev_get_holders(char *syspath)
+{
+	char *path = NULL;
+	struct dirent **holds = NULL;
+	int n, i;
+	int count = 0;
+
+	path = calloc(1, PATH_MAX);
+	if (!path)
+		return 0;
+	snprintf(path, PATH_MAX, "%s/holders", syspath);
+
+	n = scandir(path, &holds, trans_filter, alphasort);
+	for (i = 0; i < n; i++) {
+		char *newpath;
+		char *rp;
+
+		newpath = calloc(1, PATH_MAX);
+		if (!newpath)
+			continue;
+		snprintf(newpath, PATH_MAX, "%s/%s", path, holds[i]->d_name);
+
+		free(holds[i]);
+		rp = realpath(newpath, NULL);
+		if (rp)
+			count += count_device_users(rp);
+		free(newpath);
+		free(rp);
+	}
+	free(path);
+	free(holds);
+	return count;
+}
+
+static int count_device_users(char *syspath)
+{
+	int count = 0;
+	count += blockdev_check_mnts(syspath);
+	count += blockdev_get_partitions(syspath);
+	count += blockdev_get_holders(syspath);
+	return count;
+};
+
+static void device_in_use(void *data, int host_no, int target, int lun)
+{
+	char *syspath = NULL;
+	char *devname = NULL;
+	int *count = data;
+
+	devname = iscsi_sysfs_get_blockdev_from_lun(host_no, target, lun);
+	if (!devname)
+		goto out;
+	syspath = calloc(1, PATH_MAX);
+	if (!syspath)
+		goto out;
+	snprintf(syspath, PATH_MAX, "/sys/class/block/%s", devname);
+	*count += count_device_users(syspath);
+out:
+	free(syspath);
+	free(devname);
+}
+
+static int session_in_use(int sid)
+{
+	int host_no = -1, err = 0;
+	int count = 0;
+
+	if (libmount_init()) {
+		log_error("Failed to initialize libmount, "
+			  "not checking for active mounts on session [%d].\n", sid);
+		return 0;
+	}
+
+	host_no = iscsi_sysfs_get_host_no_from_sid(sid, &err);
+	if (!err)
+		iscsi_sysfs_for_each_device(&count, host_no, sid, device_in_use);
+
+	libmount_cleanup();
+	return count;
+}
+
 int session_logout_task(int sid, queue_task_t *qtask)
 {
 	iscsi_session_t *session;
 	iscsi_conn_t *conn;
 	int rc = ISCSI_SUCCESS;
+	char *safe;

 	session = session_find_by_sid(sid);
 	if (!session) {
 <at>  <at>  -2167,6 +2357,15  <at>  <at>  invalid_state:
 		return ISCSI_ERR_INTERNAL;
 	}

+	safe = cfg_get_string_param(dconfig->config_file, "iscsid.safe_logout");
+	if (safe && !strcmp(safe, "Yes") && session_in_use(sid)) {
+		log_error("Session is actively in use for mounted storage, "
+			  "and iscsid.safe_logout is configured.\n");
+		free(safe);
+		return ISCSI_ERR_BUSY;
+	}
+	free(safe);
+
 	/* FIXME: logout all active connections */
 	conn = &session->conn[0];
 	if (conn->logout_qtask)
diff --git a/usr/sysfs.c b/usr/sysfs.c
index d00c925..bbb00c0 100644
--- a/usr/sysfs.c
+++ b/usr/sysfs.c
 <at>  <at>  -709,3 +709,43  <at>  <at>  int sysfs_set_param(char *id, char *subsys, char *attr_name,
 	close(fd);
 	return rc;
 }
+
+char *sysfs_get_uevent_field(const char *path, const char *field)
+{
+	char *uevent_path = NULL;
+	FILE *f = NULL;
+	char *line, buffer[1024];
+	char *ff, *d;
+	char *out = NULL;
+
+	uevent_path = calloc(1, PATH_MAX);
+	if (!uevent_path)
+		return NULL;
+	snprintf(uevent_path, PATH_MAX, "%s/uevent", path);
+
+	f = fopen(uevent_path, "r");
+	if (!f)
+		goto out;
+	while ((line = fgets(buffer, sizeof (buffer), f))) {
+		ff = strtok(line, "=");
+		d = strtok(NULL, "\n");
+		if (strcmp(ff, field))
+			continue;
+		out = strdup(d);
+		break;
+	}
+	fclose(f);
+out:
+	free(uevent_path);
+	return out;
+}
+
+char *sysfs_get_uevent_devtype(const char *path)
+{
+	return sysfs_get_uevent_field(path, "DEVTYPE");
+}
+
+char *sysfs_get_uevent_devname(const char *path)
+{
+	return sysfs_get_uevent_field(path, "DEVNAME");
+}
diff --git a/usr/sysfs.h b/usr/sysfs.h
index 304dbbf..462060e 100644
--- a/usr/sysfs.h
+++ b/usr/sysfs.h
 <at>  <at>  -66,4 +66,8  <at>  <at>  extern int sysfs_get_uint16(char *id, char *subsys, char *param,
 extern int sysfs_set_param(char *id, char *subsys, char *attr_name,
 			   char *write_buf, ssize_t buf_size);

+extern char *sysfs_get_uevent_field(const char *path, const char *field);
+extern char *sysfs_get_uevent_devtype(const char *path);
+extern char *sysfs_get_uevent_devname(const char *path);
+
 #endif
-- 
2.1.0

--

-- 
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe@...
To post to this group, send email to open-iscsi@...
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

The Lee-Man | 30 Jan 23:09 2015
Picon

iscsid stop using system not working correctly

Hi Mike:

Just a heads up that stopping the open-iscsi iscsid daemon using systemd doesn't seem to be working correctly, at least not on SUSE SLE 12.

When I have one or more sessions present, and their startup value is set to "manual", when I try to stop the iscsid service, I get:

    # systemctl stop iscsid.service
    Job for iscsid.service canceled.

And a "ps" shows that iscsid is still running, but under a new process id. And, at times, I see that "iscsiadm -k 0 2" is hung.

Note that my systemd iscsid.service unit file looks like:

    [Unit]
    Description=Open-iSCSI
    Documentation=man:iscsid(8) man:iscsiuio(8) man:iscsiadm(8)
    DefaultDependencies=no
    After=network.target iscsiuio.service
    Before=remote-fs-pre.target

    [Service]
    Type=simple
    ExecStart=/sbin/iscsid -f
    ExecStop=/sbin/iscsiadm -k 0 2

    [Install]
    WantedBy=multi-user.target
    Also=iscsid.socket

While trying to track down this problem, I found a couple of issues:

1. I don't know what the "2" is on the "iscsiadm -k 0 2" is for. I see that command in systemd unit file in your master branch, and I see that other distributions use that, but the "2" seems to be ignored.

2. It looks like the "iscsiadm -k 0" is stopping the iscsi daemon, but that systemd is restarting it! I'm guessing this because (a) I get the "cancelled" message, and (b) iscsid is still running, but as a new process, i.e. it's been restarted.

I thought perhaps the problem was related to running iscsid as a "simple" service, in the foreground, but changing Type to forking and removing the "-f" from the iscsid command line did not change anything.

Then I simply commented out the "ExecStop" line, and iscsid now shutdowns correctly. This, I believe, is because systemd is shutting it down by sending first SIGHUP then SIGKILL, as per "man systemd.kill".

Before I comment out the "ExecStop=" for all SLE 12 users, I wondered if you've heard of any problems along these lines.

--
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To post to this group, send email to open-iscsi-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
Or Gerlitz | 21 Jan 15:19 2015

setting declarative params after login?

Hi Mike,

Long time.. hope you are all well after EOY holidays and with energy to 
for the MCS debates  <at>  LSF...

So back in upstream commit ea05be3ff043efd44256283d968fa1bb9a371568 
"iscsi tools: set non negotiated params early" we are
doing set_param towards the kernel/transport before sending the login 
request.

Running with latest upstream (commit 
76a441ba0dc0071a19daeac456aa898889437efd) we did dump of all set_params 
done towards iser and I see that Max Recv DSL and friends are set after 
sending the login, in both discovery and normal sessions (see below), is 
that a bug? aren't they declarative?

Or.

Discovery session param 43 value 0

scsi host11: iSCSI Initiator over iSER
iser: iscsi_iser_conn_bind: binding iscsi conn ffff880212309ca8 to 
iser_conn ffff880216838000
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 18 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 26 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 27 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 28 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 35 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 30 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 31 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 32 buf iser
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 34 buf 
iqn.1994-05.com.redhat:afa6a392a57a
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 43 buf 1
iser: iscsi_iser_mtask_xmit: mtask xmit [cid 0 itt 0x0]
iser: iser_send_control: op 43 dsl f4, posting login rx buffer
iser: iser_post_rx_bufs: req op 43 flags 87
iser: iser_post_rx_bufs: Discovery session, re-using login RX buffer
iser: iser_cq_tasklet_fn: got 1 completions
iser: iser_rcv_completion: op 0x23 itt 0x0 dlen 142
iser: iser_cq_tasklet_fn: got 1 completions
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 0 buf 8192
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 
ISCSI_PARAM_MAX_RECV_DLENGTH size 8192
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 1 buf 8192
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 4 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 5 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 6 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 7 buf 262144
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 8 buf 16776192
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 9 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 10 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 11 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 14 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 16 buf 1

Normal session param 43 value 0

iser: iscsi_iser_conn_bind: binding iscsi conn ffff8800c04394a8 to 
iser_conn ffff8800c147a000
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 15 buf 
iqn.2001-04.com.r-dcs53-tgt-2
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 17 buf 
192.168.20.18
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 18 buf 3260
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 26 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 27 buf 15
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 28 buf 30
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 35 buf 30
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 30 buf 5
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 31 buf 5
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 32 buf iser
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 34 buf 
iqn.1994-05.com.redhat:afa6a392a57a
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 43 buf 0
iser: iscsi_iser_mtask_xmit: mtask xmit [cid 0 itt 0x0]
iser: iser_send_control: op 43 dsl 1d4, posting login rx buffer
iser: iser_post_rx_bufs: req op 43 flags 87
iser: iser_post_rx_bufs: Normal session, posting batch of RX 32 buffers
iser: iser_cq_tasklet_fn: got 1 completions
iser: iser_rcv_completion: op 0x23 itt 0x0 dlen 355
iser: iser_cq_tasklet_fn: got 1 completions
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 0 buf 262144
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 
ISCSI_PARAM_MAX_RECV_DLENGTH size 262144
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 1 buf 8192
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 4 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 5 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 6 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 7 buf 65536
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 8 buf 262144
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 9 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 10 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 11 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 14 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 16 buf 1

--

-- 
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe@...
To post to this group, send email to open-iscsi@...
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Or Gerlitz | 21 Jan 15:19 2015

setting declarative params after login?

Hi Mike,

Long time.. hope you are all well after EOY holidays and with energy to 
for the MCS debates  <at>  LSF...

So back in upstream commit ea05be3ff043efd44256283d968fa1bb9a371568 
"iscsi tools: set non negotiated params early" we are
doing set_param towards the kernel/transport before sending the login 
request.

Running with latest upstream (commit 
76a441ba0dc0071a19daeac456aa898889437efd) we did dump of all set_params 
done towards iser and I see that Max Recv DSL and friends are set after 
sending the login, in both discovery and normal sessions (see below), is 
that a bug? aren't they declarative?

Or.

Discovery session patam 43 value 0

scsi host11: iSCSI Initiator over iSER
iser: iscsi_iser_conn_bind: binding iscsi conn ffff880212309ca8 to 
iser_conn ffff880216838000
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 18 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 26 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 27 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 28 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 35 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 30 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 31 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 32 buf iser
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 34 buf 
iqn.1994-05.com.redhat:afa6a392a57a
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 43 buf 1
iser: iscsi_iser_mtask_xmit: mtask xmit [cid 0 itt 0x0]
iser: iser_send_control: op 43 dsl f4, posting login rx buffer
iser: iser_post_rx_bufs: req op 43 flags 87
iser: iser_post_rx_bufs: Discovery session, re-using login RX buffer
iser: iser_cq_tasklet_fn: got 1 completions
iser: iser_rcv_completion: op 0x23 itt 0x0 dlen 142
iser: iser_cq_tasklet_fn: got 1 completions
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 0 buf 8192
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 
ISCSI_PARAM_MAX_RECV_DLENGTH size 8192
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 1 buf 8192
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 4 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 5 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 6 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 7 buf 262144
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 8 buf 16776192
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 9 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 10 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 11 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 14 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 16 buf 1

Normal session patam 43 value 0
iser: iscsi_iser_conn_bind: binding iscsi conn ffff8800c04394a8 to 
iser_conn ffff8800c147a000
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 15 buf 
iqn.2001-04.com.r-dcs53-tgt-2
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 17 buf 
192.168.20.18
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 18 buf 3260
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 26 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 27 buf 15
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 28 buf 30
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 35 buf 30
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 30 buf 5
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 31 buf 5
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 32 buf iser
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 34 buf 
iqn.1994-05.com.redhat:afa6a392a57a
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 43 buf 0
iser: iscsi_iser_mtask_xmit: mtask xmit [cid 0 itt 0x0]
iser: iser_send_control: op 43 dsl 1d4, posting login rx buffer
iser: iser_post_rx_bufs: req op 43 flags 87
iser: iser_post_rx_bufs: Normal session, posting batch of RX 32 buffers
iser: iser_cq_tasklet_fn: got 1 completions
iser: iser_rcv_completion: op 0x23 itt 0x0 dlen 355
iser: iser_cq_tasklet_fn: got 1 completions
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 0 buf 262144
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 
ISCSI_PARAM_MAX_RECV_DLENGTH size 262144
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 1 buf 8192
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 4 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 5 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 6 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 7 buf 65536
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 8 buf 262144
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 9 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 10 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 11 buf 0
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 14 buf 1
iser: iscsi_iser_set_param: iscsi_iser_set_param called for 16 buf 1

--

-- 
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe@...
To post to this group, send email to open-iscsi@...
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.


Gmane