aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2018-05-01sunrpc: Fix latency trace point crashesChuck Lever1-10/+6
If the rpc_task survived longer than the transport, task->tk_xprt points to freed memory by the time rpc_count_iostats_metrics runs. Replace the references to task->tk_xprt with references to the task's tk_client. Reported-by: [email protected] Fixes: 40bf7eb304b5 ('sunrpc: Add static trace point to report ...') Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-05-01xprtrdma: Fix list corruption / DMAR errors during MR recoveryChuck Lever4-11/+10
The ro_release_mr methods check whether mr->mr_list is empty. Therefore, be sure to always use list_del_init when removing an MR linked into a list using that field. Otherwise, when recovering from transport failures or device removal, list corruption can result, or MRs can get mapped or unmapped an odd number of times, resulting in IOMMU-related failures. In general this fix is appropriate back to v4.8. However, code changes since then make it impossible to apply this patch directly to stable kernels. The fix would have to be applied by hand or reworked for kernels earlier than v4.16. Backport guidance -- there are several cases: - When creating an MR, initialize mr_list so that using list_empty on an as-yet-unused MR is safe. - When an MR is being handled by the remote invalidation path, ensure that mr_list is reinitialized when it is removed from rl_registered. - When an MR is being handled by rpcrdma_destroy_mrs, it is removed from mr_all, but it may still be on an rl_registered list. In that case, the MR needs to be removed from that list before being released. - Other cases are covered by using list_del_init in rpcrdma_mr_pop. Fixes: 9d6b04097882 ('xprtrdma: Place registered MWs on a ... ') Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFS: advance nfs_entry cookie only after decoding completes successfullyFrank Sorenson2-4/+10
In nfs[34]_decode_dirent, the cookie is advanced as soon as it is read, but decoding may still fail later in the function, returning an error. Because the cookie has been advanced, the failing entry is not re-requested from the server, resulting in a missing directory entry. In addition, nfs v3 and v4 read the cookie at different locations in the xdr_stream, so the behavior of the two can be inconsistent. Fix these by reading the cookie into a temporary variable, and only advancing the cookie once the entire entry has been decoded from the xdr_stream successfully. Signed-off-by: Frank Sorenson <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv3/acl: forget acl cache after setattrchendt1-1/+4
Sync of ACL with std permissions fail，We need to forget the ACL cache after setattr. Reproduction: #!/bin/bash touch testfile cat <<EOF >testfile #!/bin/bash echo "Test was executed" EOF chmod u=rwx testfile chmod g=rw- testfile chmod o=r-- testfile chacl u::r--,g::rwx,o:rw- testfile chmod u+w testfile ls -l testfile chacl -l testfile Output： -rw-rwxrw- 1 root root 0 Mar 28 05:29 testfile testfile [u::r--,g::rwx,o::rw-] Signed-off-by: chendt.fnst <[email protected]> Reviewed-by: Benjamin Coddington <[email protected]> Reviewed-by: Kinglong Mee <Kinglong Mee> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv4.1: Fix exclusive createTrond Myklebust1-16/+33
When we use EXCLUSIVE4_1 mode, the server returns an attribute mask where all the bits indicate which attributes were set, and where the verifier was stored. In order to figure out which attribute we have to resend, we need to clear out the attributes that are set in exclcreat_bitmask. Signed-off-by: Trond Myklebust <[email protected]> [Anna: Fixed typo NFS4_CREATE_EXCLUSIVE4 -> NFS4_CREATE_EXCLUSIVE] Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv4: Declare the size up to date after it was set.Trond Myklebust2-0/+2
When we've changed the file size, then ensure we declare it to be up to date in the inode attributes. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10nfs: Use ida_simple APIMatthew Wilcox1-12/+10
Allocate the owner_id when we allocate the state and free it when we free the state. That lets us get rid of a gnarly ida_pre_get() / ida_get_new() loop. Signed-off-by: Matthew Wilcox <[email protected]> Reviewed-by: Jeff Layton <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv4: Fix the nfs_inode_set_delegation() argumentsTrond Myklebust3-19/+34
Neither nfs_inode_set_delegation() nor nfs_inode_reclaim_delegation() are generic code. They have no business delving into NFSv4 OPEN xdr structures, so let's replace the "struct nfs_openres" parameter. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv4: Clean up CB_GETATTR encodingTrond Myklebust1-29/+8
Replace the open coded bitmap implementation with a generic one. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv4: Don't ask for attributes when ACCESS is protected by a delegationTrond Myklebust2-7/+12
If we hold a delegation, then the results of the ACCESS call are protected anyway. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv4: Add a helper to encode/decode struct timespecTrond Myklebust1-15/+30
Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv4: Clean up encode_attrsTrond Myklebust1-15/+2
Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv4; Clean up XDR encoding of type bitmap4Trond Myklebust2-94/+135
Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv4: Allow GFP_NOIO sleeps in decode_attr_owner/decode_attr_groupTrond Myklebust1-2/+2
Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10SUNRPC: Add a helper for encoding opaque data inlineTrond Myklebust1-0/+25
Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10SUNRPC: Add helpers for decoding opaque and string typesTrond Myklebust2-0/+88
Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv4: Ignore change attribute invalidations if we hold a delegationTrond Myklebust1-0/+3
Don't bother even recording an invalid change attribute if we hold a delegation since we already know the state of our attribute cache. We can rely on the fact that we will pick up a copy from the server when we return the delegation. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFS: More fine grained attribute trackingTrond Myklebust5-54/+94
Currently, if the NFS_INO_INVALID_ATTR flag is set, for instance by a call to nfs_post_op_update_inode_locked(), then it will not be cleared until all the attributes have been revalidated. This means, for instance, that NFSv4 writes will always force a full attribute revalidation. Track the ctime, mtime, size and change attribute separately from the other attributes so that we can have nfs_post_op_update_inode_locked() set them correctly, and later have the cache consistency bitmask be able to clear them. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFS: Don't force unnecessary cache invalidation in nfs_update_inode()Trond Myklebust1-4/+3
If we managed to revalidate all the attributes, then there is no reason to mark them as invalid again. We do, however want to ensure that we set nfsi->attrtimeo correctly. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFS: Don't redirty the attribute cache in nfs_wcc_update_inode()Trond Myklebust1-10/+2
If we received weak cache consistency data from the server, then those attributes are up to date, and there is no reason to mark them as dirty in the attribute cache. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFS: Don't force a revalidation of all attributes if change is missingTrond Myklebust1-1/+4
Even if the change attribute is missing, it is still OK to mark the other attributes as being up to date. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFS: Convert NFS_INO_INVALID flags to unsigned longTrond Myklebust1-8/+8
The cache validity attribute is unsigned long, so make sure that the flags are too. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv4: Don't return the delegation when not needed by NFSv4.x (x>0)Trond Myklebust3-6/+28
Starting with NFSv4.1, the server is able to deduce the client id from the SEQUENCE op which means it can always figure out whether or not the client is holding a delegation on a file that is being changed. For that reason, RFC5661 does not require a delegation to be unconditionally recalled on operations such as SETATTR, RENAME, or REMOVE. Note that for now, we continue to return READ delegations since that is still expected by the Linux knfsd server. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFS: Remove the unused return_delegation() callbackTrond Myklebust4-16/+0
Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFS: Move the delegation return down into _nfs4_do_setattr()Trond Myklebust2-5/+4
Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFS: Add a delegation return into nfs4_proc_unlink_setup()Trond Myklebust5-7/+10
Ensure that when we do finally delete the file, then we return the delegation. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFS: Move delegation recall into the NFSv4 callback for rename_setup()Trond Myklebust6-14/+20
Move the delegation recall out of the generic code, and into the NFSv4 specific callback. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFS: Move the delegation return down into nfs4_proc_remove()Trond Myklebust5-12/+29
Move the delegation return out of generic code and down into the NFSv4 specific unlink code. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFS: Move the delegation return down into nfs4_proc_link()Trond Myklebust2-2/+2
Move the delegation return out of generic code. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10NFSv4: Fix nfs4_return_incompatible_delegationTrond Myklebust1-0/+1
The 'fmode' argument can take an FMODE_EXEC value, which we want to filter out before comparing to the delegation type. Signed-off-by: Trond Myklebust <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10xprtrdma: Fix corner cases when handling device removalChuck Lever1-4/+9
Michal Kalderon has found some corner cases around device unload with active NFS mounts that I didn't have the imagination to test when xprtrdma device removal was added last year. - The ULP device removal handler is responsible for deallocating the PD. That wasn't clear to me initially, and my own testing suggested it was not necessary, but that is incorrect. - The transport destruction path can no longer assume that there is a valid ID. - When destroying a transport, ensure that ib_free_cq() is not invoked on a CQ that was already released. Reported-by: Michal Kalderon <[email protected]> Fixes: bebd031866ca ("xprtrdma: Support unplugging an HCA from ...") Signed-off-by: Chuck Lever <[email protected]> Cc: [email protected] # v4.12+ Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10nfs4: wake any lock waiters on successful RECLAIM_COMPLETEJeff Layton1-10/+16
If we have a RECLAIM_COMPLETE with a populated cl_lock_waitq, then that implies that a reconnect has occurred. Since we can't expect a CB_NOTIFY_LOCK callback at that point, just wake up the entire queue so that all the tasks can re-poll for their locks. Signed-off-by: Jeff Layton <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10nfs4: don't compare clientid in nfs4_wake_lock_waiterJeff Layton1-4/+2
The task is expected to sleep for a while here, and it's possible that a new EXCHANGE_ID has occurred in the interim, and we were assigned a new clientid. Since this is a per-client list, there isn't a lot of value in vetting the clientid on the incoming request. Signed-off-by: Jeff Layton <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10nfs4: always reset notified flag to false before repolling for lockJeff Layton1-0/+1
We may get a notification and lose the race to another client. Ensure that we wait again for a notification in that case. Signed-off-by: Jeff Layton <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10sunrpc: Add static trace point to report result of RPC pingChuck Lever2-0/+27
This information can help track down local misconfiguration issues as well as network partitions and unresponsive servers. There are several ways to send a ping, and with transport multi- plexing, the exact rpc_xprt that is used is sometimes not known by the upper layer. The rpc_xprt pointer passed to the trace point call also has to be RCU-safe. I found a spot inside the client FSM where an rpc_xprt pointer is always available and safe to use. Suggested-by: Bill Baker <[email protected]> Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10sunrpc: Add static trace point to report RPC latency statsChuck Lever2-5/+56
Introduce a low-overhead mechanism to report information about latencies of individual RPCs. The goal is to enable user space to filter the trace record for latency outliers, or build histograms, etc. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10sunrpc: Simplify synopsis of some trace pointsChuck Lever3-25/+27
Clean up: struct rpc_task carries a pointer to a struct rpc_clnt, and in fact task->tk_client is always what is passed into trace points that are already passing @task. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10SUNRPC: Make num_reqs a non-atomic integerChuck Lever2-9/+10
If recording xprt->stat.max_slots is moved into xprt_alloc_slot, then xprt->num_reqs is never manipulated outside xprt->reserve_lock. There's no longer a need for xprt->num_reqs to be atomic. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10SUNRPC: Make RTT measurement more precise (Send)Chuck Lever3-1/+4
Some RPC transports have more overhead in their send_request callouts than others. For example, for RPC-over-RDMA: - Marshaling an RPC often has to DMA map the RPC arguments - Registration methods perform memory registration as part of marshaling To capture just server and network latencies more precisely: when sending a Call, capture the rq_xtime timestamp _after_ the transport header has been marshaled. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10SUNRPC: Make RTT measurement more precise (Receive)Chuck Lever1-2/+3
Some RPC transports have more overhead in their reply handlers than others. For example, for RPC-over-RDMA: - RPC completion has to wait for memory invalidation, which is not a part of the server/network round trip - Recently a context switch was introduced into the reply handler, which further artificially inflates the measure of RPC RTT To capture just server and network latencies more precisely: when receiving a reply, compute the RTT as soon as the XID is recognized rather than at RPC completion time. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10SUNRPC: Move xprt_update_rtt callsiteChuck Lever3-3/+10
Since commit 33849792cbcd ("xprtrdma: Detect unreachable NFS/RDMA servers more reliably"), the xprtrdma transport now has a ->timer callout. But xprtrdma does not need to compute RTT data, only UDP needs that. Move the xprt_update_rtt call into the UDP transport implementation. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10xprtrdma: Move creation of rl_rdmabuf to rpcrdma_create_reqChuck Lever3-34/+12
Refactor: Both rpcrdma_create_req call sites have to allocate the buffer where the transport header is built, so just move that allocation into rpcrdma_create_req. This buffer is a fixed size. There's no needed information available in call_allocate that is not also available when the transport is created. The original purpose for allocating these buffers on demand was to reduce the possibility that an allocation failure during transport creation will hork the mount operation during low memory scenarios. Some relief for this rare possibility is coming up in the next few patches. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10xprtrdma: Chain Send to FastReg WRsChuck Lever4-18/+49
With FRWR, the client transport can perform memory registration and post a Send with just a single ib_post_send. This reduces contention between the send_request path and the Send Completion handlers, and reduces the overhead of registering a chunk that has multiple segments. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10xprtrdma: "Support" call-only RPCsChuck Lever3-6/+13
RPC-over-RDMA version 1 credit accounting relies on there being a response message for every RPC Call. This means that RPC procedures that have no reply will disrupt credit accounting, just in the same way as a retransmit would (since it is sent because no reply has arrived). Deal with the "no reply" case the same way. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10xprtrdma: Reduce number of MRs created by rpcrdma_mrs_createChuck Lever1-1/+1
Create fewer MRs on average. Many workloads don't need as many as 32 MRs, and the transport can now quickly restock the MR free list. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10xprtrdma: ->send_request returns -EAGAIN when there are no free MRsChuck Lever5-13/+27
Currently, when the MR free list is exhausted during marshaling, the RPC/RDMA transport places the RPC task on the delayq, which forces a wait for HZ >> 2 before the marshal and send is retried. With this change, the transport now places such an RPC task on the pending queue, and wakes it just as soon as more MRs have been created. Creating more MRs typically takes less than a millisecond, and this waking mechanism is less deadlock-prone. Moreover, the waiting RPC task is holding the transport's write lock, which blocks the transport from sending RPCs. Therefore faster recovery from MR exhaustion is desirable. This is the same mechanism that the TCP transport utilizes when handling write buffer space exhaustion. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10xprtrdma: Remove xprt-specific connect cookieChuck Lever3-6/+3
Clean up: The generic rq_connect_cookie is sufficient to detect RPC Call retransmission. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10xprtrdma: Remove arbitrary limit on initiator depthChuck Lever1-5/+2
Clean up: We need to check only that the value does not exceed the range of the u8 field it's going into. Signed-off-by: Chuck Lever <[email protected]> Signed-off-by: Anna Schumaker <[email protected]>
2018-04-10xprtrdma: Fix latency regression on NUMA NFS/RDMA clientsChuck Lever3-4/+1
With v4.15, on one of my NFS/RDMA clients I measured a nearly doubling in the latency of small read and write system calls. There was no change in server round trip time. The extra latency appears in the whole RPC execution path. "git bisect" settled on commit ccede7598588 ("xprtrdma: Spread reply processing over more CPUs") . After some experimentation, I found that leaving the WQ bound and allowing the scheduler to pick the dispatch CPU seems to eliminate the long latencies, and it does not introduce any new regressions. The fix is implemented by reverting only the part of commit ccede7598588 ("xprtrdma: Spread reply processing over more CPUs") that dispatches RPC replies specifically on the CPU where the matching RPC call was made. Interestingly, saving the CPU number and later queuing reply processing there was effective _only_ for a NFS READ and WRITE request. On my NUMA client, in-kernel RPC reply processing for asynchronous RPCs was dispatched on the same CPU where the RPC call was made, as expected. However synchronous RPCs seem to get their reply dispatched on some other CPU than where the call was placed, every time. Fixes: ccede7598588 ("xprtrdma: Spread reply processing over ... ") Signed-off-by: Chuck Lever <[email protected]> Cc: [email protected] # v4.15+ Signed-off-by: Anna Schumaker <[email protected]>
2018-03-04Linux 4.16-rc4Linus Torvalds1-1/+1
Powered by cgit, Gruvboxed by Blaster4385.