| Benny Halevy | 3ef1728 | 2009-04-03 08:29:20 +0300 | [diff] [blame] | 1 | NFSv4.1 Server Implementation | 
 | 2 |  | 
 | 3 | Server support for minorversion 1 can be controlled using the | 
 | 4 | /proc/fs/nfsd/versions control file.  The string output returned | 
 | 5 | by reading this file will contain either "+4.1" or "-4.1" | 
 | 6 | correspondingly. | 
 | 7 |  | 
 | 8 | Currently, server support for minorversion 1 is disabled by default. | 
 | 9 | It can be enabled at run time by writing the string "+4.1" to | 
 | 10 | the /proc/fs/nfsd/versions control file.  Note that to write this | 
 | 11 | control file, the nfsd service must be taken down.  Use your user-mode | 
 | 12 | nfs-utils to set this up; see rpc.nfsd(8) | 
 | 13 |  | 
| J. Bruce Fields | 285a0f0 | 2009-09-20 17:01:33 -0400 | [diff] [blame] | 14 | (Warning: older servers will interpret "+4.1" and "-4.1" as "+4" and | 
 | 15 | "-4", respectively.  Therefore, code meant to work on both new and old | 
 | 16 | kernels must turn 4.1 on or off *before* turning support for version 4 | 
 | 17 | on or off; rpc.nfsd does this correctly.) | 
 | 18 |  | 
| Benny Halevy | 3ef1728 | 2009-04-03 08:29:20 +0300 | [diff] [blame] | 19 | The NFSv4 minorversion 1 (NFSv4.1) implementation in nfsd is based | 
| J. Bruce Fields | 73834d6 | 2010-01-20 17:17:04 -0500 | [diff] [blame] | 20 | on RFC 5661. | 
| Benny Halevy | 3ef1728 | 2009-04-03 08:29:20 +0300 | [diff] [blame] | 21 |  | 
 | 22 | From the many new features in NFSv4.1 the current implementation | 
 | 23 | focuses on the mandatory-to-implement NFSv4.1 Sessions, providing | 
 | 24 | "exactly once" semantics and better control and throttling of the | 
 | 25 | resources allocated for each client. | 
 | 26 |  | 
 | 27 | Other NFSv4.1 features, Parallel NFS operations in particular, | 
 | 28 | are still under development out of tree. | 
 | 29 | See http://wiki.linux-nfs.org/wiki/index.php/PNFS_prototype_design | 
 | 30 | for more information. | 
 | 31 |  | 
| J. Bruce Fields | 285a0f0 | 2009-09-20 17:01:33 -0400 | [diff] [blame] | 32 | The current implementation is intended for developers only: while it | 
 | 33 | does support ordinary file operations on clients we have tested against | 
 | 34 | (including the linux client), it is incomplete in ways which may limit | 
 | 35 | features unexpectedly, cause known bugs in rare cases, or cause | 
 | 36 | interoperability problems with future clients.  Known issues: | 
 | 37 |  | 
 | 38 | 	- gss support is questionable: currently mounts with kerberos | 
 | 39 | 	  from a linux client are possible, but we aren't really | 
 | 40 | 	  conformant with the spec (for example, we don't use kerberos | 
 | 41 | 	  on the backchannel correctly). | 
 | 42 | 	- no trunking support: no clients currently take advantage of | 
| J. Bruce Fields | 03d6a74 | 2009-09-22 11:09:12 -0400 | [diff] [blame] | 43 | 	  trunking, but this is a mandatory feature, and its use is | 
| J. Bruce Fields | 285a0f0 | 2009-09-20 17:01:33 -0400 | [diff] [blame] | 44 | 	  recommended to clients in a number of places.  (E.g. to ensure | 
 | 45 | 	  timely renewal in case an existing connection's retry timeouts | 
| J. Bruce Fields | 73834d6 | 2010-01-20 17:17:04 -0500 | [diff] [blame] | 46 | 	  have gotten too long; see section 8.3 of the RFC.) | 
| J. Bruce Fields | 285a0f0 | 2009-09-20 17:01:33 -0400 | [diff] [blame] | 47 | 	  Therefore, lack of this feature may cause future clients to | 
 | 48 | 	  fail. | 
 | 49 | 	- Incomplete backchannel support: incomplete backchannel gss | 
 | 50 | 	  support and no support for BACKCHANNEL_CTL mean that | 
 | 51 | 	  callbacks (hence delegations and layouts) may not be | 
 | 52 | 	  available and clients confused by the incomplete | 
 | 53 | 	  implementation may fail. | 
 | 54 | 	- Server reboot recovery is unsupported; if the server reboots, | 
 | 55 | 	  clients may fail. | 
 | 56 | 	- We do not support SSV, which provides security for shared | 
 | 57 | 	  client-server state (thus preventing unauthorized tampering | 
 | 58 | 	  with locks and opens, for example).  It is mandatory for | 
 | 59 | 	  servers to support this, though no clients use it yet. | 
 | 60 | 	- Mandatory operations which we do not support, such as | 
 | 61 | 	  DESTROY_CLIENTID, FREE_STATEID, SECINFO_NO_NAME, and | 
 | 62 | 	  TEST_STATEID, are not currently used by clients, but will be | 
 | 63 | 	  (and the spec recommends their uses in common cases), and | 
 | 64 | 	  clients should not be expected to know how to recover from the | 
 | 65 | 	  case where they are not supported.  This will eventually cause | 
 | 66 | 	  interoperability failures. | 
 | 67 |  | 
 | 68 | In addition, some limitations are inherited from the current NFSv4 | 
 | 69 | implementation: | 
 | 70 |  | 
 | 71 | 	- Incomplete delegation enforcement: if a file is renamed or | 
 | 72 | 	  unlinked, a client holding a delegation may continue to | 
 | 73 | 	  indefinitely allow opens of the file under the old name. | 
 | 74 |  | 
| Benny Halevy | 3ef1728 | 2009-04-03 08:29:20 +0300 | [diff] [blame] | 75 | The table below, taken from the NFSv4.1 document, lists | 
 | 76 | the operations that are mandatory to implement (REQ), optional | 
 | 77 | (OPT), and NFSv4.0 operations that are required not to implement (MNI) | 
 | 78 | in minor version 1.  The first column indicates the operations that | 
 | 79 | are not supported yet by the linux server implementation. | 
 | 80 |  | 
 | 81 | The OPTIONAL features identified and their abbreviations are as follows: | 
 | 82 | 	pNFS	Parallel NFS | 
 | 83 | 	FDELG	File Delegations | 
 | 84 | 	DDELG	Directory Delegations | 
 | 85 |  | 
 | 86 | The following abbreviations indicate the linux server implementation status. | 
 | 87 | 	I	Implemented NFSv4.1 operations. | 
 | 88 | 	NS	Not Supported. | 
 | 89 | 	NS*	unimplemented optional feature. | 
 | 90 | 	P	pNFS features implemented out of tree. | 
 | 91 | 	PNS	pNFS features that are not supported yet (out of tree). | 
 | 92 |  | 
 | 93 | Operations | 
 | 94 |  | 
 | 95 |    +----------------------+------------+--------------+----------------+ | 
 | 96 |    | Operation            | REQ, REC,  | Feature      | Definition     | | 
 | 97 |    |                      | OPT, or    | (REQ, REC,   |                | | 
 | 98 |    |                      | MNI        | or OPT)      |                | | 
 | 99 |    +----------------------+------------+--------------+----------------+ | 
 | 100 |    | ACCESS               | REQ        |              | Section 18.1   | | 
 | 101 | NS | BACKCHANNEL_CTL      | REQ        |              | Section 18.33  | | 
 | 102 | NS | BIND_CONN_TO_SESSION | REQ        |              | Section 18.34  | | 
 | 103 |    | CLOSE                | REQ        |              | Section 18.2   | | 
 | 104 |    | COMMIT               | REQ        |              | Section 18.3   | | 
 | 105 |    | CREATE               | REQ        |              | Section 18.4   | | 
 | 106 | I  | CREATE_SESSION       | REQ        |              | Section 18.36  | | 
 | 107 | NS*| DELEGPURGE           | OPT        | FDELG (REQ)  | Section 18.5   | | 
 | 108 |    | DELEGRETURN          | OPT        | FDELG,       | Section 18.6   | | 
 | 109 |    |                      |            | DDELG, pNFS  |                | | 
 | 110 |    |                      |            | (REQ)        |                | | 
 | 111 | NS | DESTROY_CLIENTID     | REQ        |              | Section 18.50  | | 
 | 112 | I  | DESTROY_SESSION      | REQ        |              | Section 18.37  | | 
 | 113 | I  | EXCHANGE_ID          | REQ        |              | Section 18.35  | | 
 | 114 | NS | FREE_STATEID         | REQ        |              | Section 18.38  | | 
 | 115 |    | GETATTR              | REQ        |              | Section 18.7   | | 
 | 116 | P  | GETDEVICEINFO        | OPT        | pNFS (REQ)   | Section 18.40  | | 
 | 117 | P  | GETDEVICELIST        | OPT        | pNFS (OPT)   | Section 18.41  | | 
 | 118 |    | GETFH                | REQ        |              | Section 18.8   | | 
 | 119 | NS*| GET_DIR_DELEGATION   | OPT        | DDELG (REQ)  | Section 18.39  | | 
 | 120 | P  | LAYOUTCOMMIT         | OPT        | pNFS (REQ)   | Section 18.42  | | 
 | 121 | P  | LAYOUTGET            | OPT        | pNFS (REQ)   | Section 18.43  | | 
 | 122 | P  | LAYOUTRETURN         | OPT        | pNFS (REQ)   | Section 18.44  | | 
 | 123 |    | LINK                 | OPT        |              | Section 18.9   | | 
 | 124 |    | LOCK                 | REQ        |              | Section 18.10  | | 
 | 125 |    | LOCKT                | REQ        |              | Section 18.11  | | 
 | 126 |    | LOCKU                | REQ        |              | Section 18.12  | | 
 | 127 |    | LOOKUP               | REQ        |              | Section 18.13  | | 
 | 128 |    | LOOKUPP              | REQ        |              | Section 18.14  | | 
 | 129 |    | NVERIFY              | REQ        |              | Section 18.15  | | 
 | 130 |    | OPEN                 | REQ        |              | Section 18.16  | | 
 | 131 | NS*| OPENATTR             | OPT        |              | Section 18.17  | | 
 | 132 |    | OPEN_CONFIRM         | MNI        |              | N/A            | | 
 | 133 |    | OPEN_DOWNGRADE       | REQ        |              | Section 18.18  | | 
 | 134 |    | PUTFH                | REQ        |              | Section 18.19  | | 
 | 135 |    | PUTPUBFH             | REQ        |              | Section 18.20  | | 
 | 136 |    | PUTROOTFH            | REQ        |              | Section 18.21  | | 
 | 137 |    | READ                 | REQ        |              | Section 18.22  | | 
 | 138 |    | READDIR              | REQ        |              | Section 18.23  | | 
 | 139 |    | READLINK             | OPT        |              | Section 18.24  | | 
| J. Bruce Fields | 4dc6ec0 | 2010-04-19 15:11:28 -0400 | [diff] [blame] | 140 |    | RECLAIM_COMPLETE     | REQ        |              | Section 18.51  | | 
| Benny Halevy | 3ef1728 | 2009-04-03 08:29:20 +0300 | [diff] [blame] | 141 |    | RELEASE_LOCKOWNER    | MNI        |              | N/A            | | 
 | 142 |    | REMOVE               | REQ        |              | Section 18.25  | | 
 | 143 |    | RENAME               | REQ        |              | Section 18.26  | | 
 | 144 |    | RENEW                | MNI        |              | N/A            | | 
 | 145 |    | RESTOREFH            | REQ        |              | Section 18.27  | | 
 | 146 |    | SAVEFH               | REQ        |              | Section 18.28  | | 
 | 147 |    | SECINFO              | REQ        |              | Section 18.29  | | 
 | 148 | NS | SECINFO_NO_NAME      | REC        | pNFS files   | Section 18.45, | | 
 | 149 |    |                      |            | layout (REQ) | Section 13.12  | | 
 | 150 | I  | SEQUENCE             | REQ        |              | Section 18.46  | | 
 | 151 |    | SETATTR              | REQ        |              | Section 18.30  | | 
 | 152 |    | SETCLIENTID          | MNI        |              | N/A            | | 
 | 153 |    | SETCLIENTID_CONFIRM  | MNI        |              | N/A            | | 
 | 154 | NS | SET_SSV              | REQ        |              | Section 18.47  | | 
 | 155 | NS | TEST_STATEID         | REQ        |              | Section 18.48  | | 
 | 156 |    | VERIFY               | REQ        |              | Section 18.31  | | 
 | 157 | NS*| WANT_DELEGATION      | OPT        | FDELG (OPT)  | Section 18.49  | | 
 | 158 |    | WRITE                | REQ        |              | Section 18.32  | | 
 | 159 |  | 
 | 160 | Callback Operations | 
 | 161 |  | 
 | 162 |    +-------------------------+-----------+-------------+---------------+ | 
 | 163 |    | Operation               | REQ, REC, | Feature     | Definition    | | 
 | 164 |    |                         | OPT, or   | (REQ, REC,  |               | | 
 | 165 |    |                         | MNI       | or OPT)     |               | | 
 | 166 |    +-------------------------+-----------+-------------+---------------+ | 
 | 167 |    | CB_GETATTR              | OPT       | FDELG (REQ) | Section 20.1  | | 
 | 168 | P  | CB_LAYOUTRECALL         | OPT       | pNFS (REQ)  | Section 20.3  | | 
 | 169 | NS*| CB_NOTIFY               | OPT       | DDELG (REQ) | Section 20.4  | | 
 | 170 | P  | CB_NOTIFY_DEVICEID      | OPT       | pNFS (OPT)  | Section 20.12 | | 
 | 171 | NS*| CB_NOTIFY_LOCK          | OPT       |             | Section 20.11 | | 
 | 172 | NS*| CB_PUSH_DELEG           | OPT       | FDELG (OPT) | Section 20.5  | | 
 | 173 |    | CB_RECALL               | OPT       | FDELG,      | Section 20.2  | | 
 | 174 |    |                         |           | DDELG, pNFS |               | | 
 | 175 |    |                         |           | (REQ)       |               | | 
 | 176 | NS*| CB_RECALL_ANY           | OPT       | FDELG,      | Section 20.6  | | 
 | 177 |    |                         |           | DDELG, pNFS |               | | 
 | 178 |    |                         |           | (REQ)       |               | | 
 | 179 | NS | CB_RECALL_SLOT          | REQ       |             | Section 20.8  | | 
 | 180 | NS*| CB_RECALLABLE_OBJ_AVAIL | OPT       | DDELG, pNFS | Section 20.7  | | 
 | 181 |    |                         |           | (REQ)       |               | | 
 | 182 | I  | CB_SEQUENCE             | OPT       | FDELG,      | Section 20.9  | | 
 | 183 |    |                         |           | DDELG, pNFS |               | | 
 | 184 |    |                         |           | (REQ)       |               | | 
 | 185 | NS*| CB_WANTS_CANCELLED      | OPT       | FDELG,      | Section 20.10 | | 
 | 186 |    |                         |           | DDELG, pNFS |               | | 
 | 187 |    |                         |           | (REQ)       |               | | 
 | 188 |    +-------------------------+-----------+-------------+---------------+ | 
 | 189 |  | 
 | 190 | Implementation notes: | 
 | 191 |  | 
| J. Bruce Fields | 285a0f0 | 2009-09-20 17:01:33 -0400 | [diff] [blame] | 192 | DELEGPURGE: | 
 | 193 | * mandatory only for servers that support CLAIM_DELEGATE_PREV and/or | 
 | 194 |   CLAIM_DELEG_PREV_FH (which allows clients to keep delegations that | 
 | 195 |   persist across client reboots).  Thus we need not implement this for | 
 | 196 |   now. | 
 | 197 |  | 
| Benny Halevy | 3ef1728 | 2009-04-03 08:29:20 +0300 | [diff] [blame] | 198 | EXCHANGE_ID: | 
 | 199 | * only SP4_NONE state protection supported | 
 | 200 | * implementation ids are ignored | 
 | 201 |  | 
 | 202 | CREATE_SESSION: | 
 | 203 | * backchannel attributes are ignored | 
 | 204 | * backchannel security parameters are ignored | 
 | 205 |  | 
 | 206 | SEQUENCE: | 
 | 207 | * no support for dynamic slot table renegotiation (optional) | 
 | 208 |  | 
 | 209 | nfsv4.1 COMPOUND rules: | 
 | 210 | The following cases aren't supported yet: | 
 | 211 | * Enforcing of NFS4ERR_NOT_ONLY_OP for: BIND_CONN_TO_SESSION, CREATE_SESSION, | 
 | 212 |   DESTROY_CLIENTID, DESTROY_SESSION, EXCHANGE_ID. | 
 | 213 | * DESTROY_SESSION MUST be the final operation in the COMPOUND request. | 
 | 214 |  | 
| Andy Adamson | ddc04fd | 2009-09-23 21:32:21 -0400 | [diff] [blame] | 215 | Nonstandard compound limitations: | 
 | 216 | * No support for a sessions fore channel RPC compound that requires both a | 
 | 217 |   ca_maxrequestsize request and a ca_maxresponsesize reply, so we may | 
 | 218 |   fail to live up to the promise we made in CREATE_SESSION fore channel | 
 | 219 |   negotiation. | 
 | 220 | * No more than one IO operation (read, write, readdir) allowed per | 
 | 221 |   compound. |