Skip to content
  • James Smart's avatar
    nvmet_fc: add defer_req callback for deferment of cmd buffer return · 0fb228d3
    James Smart authored
    
    
    At queue creation, the transport allocates a local job struct
    (struct nvmet_fc_fcp_iod) for each possible element of the queue.
    When a new CMD is received from the wire, a jobs struct is allocated
    from the queue and then used for the duration of the command.
    The job struct contains buffer space for the wire command iu. Thus,
    upon allocation of the job struct, the cmd iu buffer is copied to
    the job struct and the LLDD may immediately free/reuse the CMD IU
    buffer passed in the call.
    
    However, in some circumstances, due to the packetized nature of FC
    and the api of the FC LLDD which may issue a hw command to send the
    wire response, but the LLDD may not get the hw completion for the
    command and upcall the nvmet_fc layer before a new command may be
    asynchronously received on the wire. In other words, its possible
    for the initiator to get the response from the wire, thus believe a
    command slot free, and send a new command iu. The new command iu
    may be received by the LLDD and passed to the transport before the
    LLDD had serviced the hw completion and made the teardown calls for
    the original job struct. As such, there is no available job struct
    available for the new io. E.g. it appears like the host sent more
    queue elements than the queue size. It didn't based on it's
    understanding.
    
    Rather than treat this as a hard connection failure queue the new
    request until the job struct does free up. As the buffer isn't
    copied as there's no job struct, a special return value must be
    returned to the LLDD to signify to hold off on recycling the cmd
    iu buffer.  And later, when a job struct is allocated and the
    buffer copied, a new LLDD callback is introduced to notify the
    LLDD and allow it to recycle it's command iu buffer.
    
    Signed-off-by: default avatarJames Smart <james.smart@broadcom.com>
    Reviewed-by: default avatarJohannes Thumshirn <jthumshirn@suse.de>
    Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
    0fb228d3