{"action":"created","issue":{"url":"https://api.github.com/repos/wesbland/mpi-forum-historic/issues/45","repository_url":"https://api.github.com/repos/wesbland/mpi-forum-historic","labels_url":"https://api.github.com/repos/wesbland/mpi-forum-historic/issues/45/labels{/name}","comments_url":"https://api.github.com/repos/wesbland/mpi-forum-historic/issues/45/comments","events_url":"https://api.github.com/repos/wesbland/mpi-forum-historic/issues/45/events","html_url":"https://github.com/wesbland/mpi-forum-historic/issues/45","id":166670686,"number":45,"title":"Send Buffer Access","user":{"login":"wesbland","id":1561207,"avatar_url":"https://avatars.githubusercontent.com/u/1561207?v=3","gravatar_id":"","url":"https://api.github.com/users/wesbland","html_url":"https://github.com/wesbland","followers_url":"https://api.github.com/users/wesbland/followers","following_url":"https://api.github.com/users/wesbland/following{/other_user}","gists_url":"https://api.github.com/users/wesbland/gists{/gist_id}","starred_url":"https://api.github.com/users/wesbland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wesbland/subscriptions","organizations_url":"https://api.github.com/users/wesbland/orgs","repos_url":"https://api.github.com/users/wesbland/repos","events_url":"https://api.github.com/users/wesbland/events{/privacy}","received_events_url":"https://api.github.com/users/wesbland/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"wesbland","id":1561207,"avatar_url":"https://avatars.githubusercontent.com/u/1561207?v=3","gravatar_id":"","url":"https://api.github.com/users/wesbland","html_url":"https://github.com/wesbland","followers_url":"https://api.github.com/users/wesbland/followers","following_url":"https://api.github.com/users/wesbland/following{/other_user}","gists_url":"https://api.github.com/users/wesbland/gists{/gist_id}","starred_url":"https://api.github.com/users/wesbland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wesbland/subscriptions","organizations_url":"https://api.github.com/users/wesbland/orgs","repos_url":"https://api.github.com/users/wesbland/repos","events_url":"https://api.github.com/users/wesbland/events{/privacy}","received_events_url":"https://api.github.com/users/wesbland/received_events","type":"User","site_admin":false},"assignees":[{"login":"wesbland","id":1561207,"avatar_url":"https://avatars.githubusercontent.com/u/1561207?v=3","gravatar_id":"","url":"https://api.github.com/users/wesbland","html_url":"https://github.com/wesbland","followers_url":"https://api.github.com/users/wesbland/followers","following_url":"https://api.github.com/users/wesbland/following{/other_user}","gists_url":"https://api.github.com/users/wesbland/gists{/gist_id}","starred_url":"https://api.github.com/users/wesbland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wesbland/subscriptions","organizations_url":"https://api.github.com/users/wesbland/orgs","repos_url":"https://api.github.com/users/wesbland/repos","events_url":"https://api.github.com/users/wesbland/events{/privacy}","received_events_url":"https://api.github.com/users/wesbland/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/wesbland/mpi-forum-historic/milestones/2","html_url":"https://github.com/wesbland/mpi-forum-historic/milestones/2008/10/20%20Chicago","labels_url":"https://api.github.com/repos/wesbland/mpi-forum-historic/milestones/2/labels","id":1894742,"number":2,"title":"2008/10/20 Chicago","description":"","creator":{"login":"wesbland","id":1561207,"avatar_url":"https://avatars.githubusercontent.com/u/1561207?v=3","gravatar_id":"","url":"https://api.github.com/users/wesbland","html_url":"https://github.com/wesbland","followers_url":"https://api.github.com/users/wesbland/followers","following_url":"https://api.github.com/users/wesbland/following{/other_user}","gists_url":"https://api.github.com/users/wesbland/gists{/gist_id}","starred_url":"https://api.github.com/users/wesbland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wesbland/subscriptions","organizations_url":"https://api.github.com/users/wesbland/orgs","repos_url":"https://api.github.com/users/wesbland/repos","events_url":"https://api.github.com/users/wesbland/events{/privacy}","received_events_url":"https://api.github.com/users/wesbland/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":2,"state":"closed","created_at":"2016-07-20T18:12:50Z","updated_at":"2016-07-20T20:11:07Z","due_on":"2008-10-22T04:00:00Z","closed_at":"2016-07-20T18:12:50Z"},"comments":3,"created_at":"2016-07-20T20:11:03Z","updated_at":"2016-07-20T20:11:14Z","closed_at":null,"body":"**Originally by _erezh@microsoft.com_ on 2008-10-16 15:36:19 -0500**\n\n___\n# Remove Send Buffer Access Restrictionworkgroup list [mailto:mpi-buff_access@lists.mpi-forum.org]\n\n## April meeting straw votes (4/29/2008)\nYes: 16[[BR]]\nNo: 1[[BR]]\nAbstain: 11[[BR]]\n\n## Jul meeting straw votes (7/1/2008)\nYes: 22[[BR]]\nNo: 0[[BR]]\nDont care: 0[[BR]]\nDont have enough info: 4[[BR]]\n\n## Background:\nMPI 1.1 standard prohibits users from accessing for read their send buffer until the send operation completes. Be it access in the same thread in case of an async send operation or by another thread in the case of blocking send operation. The rational in the MPI 1.1 standard was to enable the performance for DMA engine that is not cache-coherent with the main processor.\n\n## Proposal:\nRemove the read access restriction on the send buffers.\n\n## Rational:\n\nThis restriction is counter intuitive for many programmers. It leads to programs that are not compliant with the spec, unaware of this limitation. This limitation prohibits a common usage of the async send API’s, and or the blocking sends.[[BR]]\nFor example the following code sequence is not compliant with the spec.\n```\n#!c\nvoid *buffer;\nint count,tag;\n...\nMPI_Isend(buffer, count, MPI_CHAR, 3, tag, MPI_COMM_WORLD, &request1);\nMPI_Isend(buffer, count, MPI_CHAR, 4, tag, MPI_COMM_WORLD, &request2);\n```\n\nThe email thread discussing the cache-coherency machines limitation concluded that it’s a non issue for these machines. The application should resolve the multi-threaded non cache-coherency issues anyhow for sending data.\n\n### Impact on MPI implementations:\nThis change will render implementations that modify the user buffer in-place (like byte-swap) invalid.\n\n### Impact on MPI applications:\n - no impact on applications that are compliant with the existing standard.\n - applications that are not compliant with the existing standard and are accessing the send buffer will become compliant (and portable).\n - new applications can take advantage of accessing the send buffer to improve performance (rather than copying the send buffer, or using blocking operations).\n\n### Proposed changes to the MPI document:\n*The location pointers are refering to the [MPI 2.1 Standard](http://www.mpi-forum.org/docs/mpi21-report.pdf)*\n\n----\n*Section 3.4 Page 38 Line 7*[[BR]]\n**Change:**[[BR]]\n  The send call described in Section 3.2.1 is blocking: it does not return until the message data and envelope have been safely stored away so that the sender is free to ~~access and overwrite~~ the send buffer.\n\n-*To:** *(change “access and overwrite” to “modify”)*[[BR]]\n  The send call described in Section 3.2.1 is blocking: it does not return until the message data and envelope have been safely stored away so that the sender is free to __modify__ the send buffer.\n\n----\n*Section 3.4 Page 41 Line 1*[[BR]]\n**Change:**[[BR]]\n  In a multi-threaded implementation of MPI, the system may de-schedule a thread that is blocked on a send or receive operation, and schedule another thread for execution in the same address space. In such a case it is the user’s responsibility not to ~~access or~~ modify a communication buffer until the communication completes. Otherwise, the outcome of the computation is undefined.\n\n-*To:** *(remove “access or”)*[[BR]]\n  In a multi-threaded implementation of MPI, the system may de-schedule a thread that is blocked on a send or receive operation, and schedule another thread for execution in the same address space. In such a case it is the user’s responsibility not to modify a communication buffer until the communication completes. Otherwise, the outcome of the computation is undefined.\n\n----\n*Section 3.4 Page 41 Lines 7-12*[[BR]]\n**Remove:**[[BR]]\n  ~~Rationale. We prohibit read accesses to a send buffer while it is being used, even though the send operation is not supposed to alter the content of this buffer. This may seem more stringent than necessary, but the additional restriction causes little loss of functionality and allows better performance on some systems — consider the case where data transfer is done by a DMA engine that is not cache-coherent with the main processor. (End of rationale.)~~\n\n----\n*Section 3.9 Page 71 Line 44*[[BR]]\n**Change:**[[BR]]\n  If the request is for a send with ready mode, then a matching receive should be posted before the call is made. The communication buffer should not be ~~accessed~~ after the call, and until the operation completes.\n\n-*To:** *(change “accessed” to “modified”)*[[BR]]\n  If the request is for a send with ready mode, then a matching receive should be posted before the call is made. The communication buffer should not be __modified__ after the call, and until the operation completes.\n\n----\n*Section 5.1 Page 131 Line 11*[[BR]]\n**Change:**[[BR]]\n  Collective routine calls can (but are not required to) return as soon as their participation in the collective communication is complete. The completion of a call indicates that the caller is now free to ~~access~~ locations in the communication buffer.\n\n-*To:** *(change “access” to “modify”)*[[BR]]\n  Collective routine calls can (but are not required to) return as soon as their participation in the collective communication is complete. The completion of a call indicates that the caller is now free to __modify__ locations in the communication buffer.\n\n----\n*Section 11.3 Page 325 Line 20-31*[[BR]]\n**Remove:**[[BR]]\n  ~~Rationale. The rule above is more lenient than for message passing, where we do not allow two concurrent sends, with overlapping send buffers. Here, we allow two concurrent puts with overlapping send buffers. The reasons for this relaxation are~~\n  1. ~~Users do not like that restriction, which is not very natural (it prohibits concurrent reads).~~\n  2. ~~Weakening the rule does not prevent efficient implementation, as far as we know.~~\n  3. ~~Weakening the rule is important for performance of RMA: we want to associate one synchronization call with as many RMA operations is possible. If puts from overlapping buffers cannot be concurrent, then we need to needlessly add synchronization points in the code.~~[[BR]]\n\n  ~~(End of rationale.)~~\n\n## Entry for the Change Log\nThe read access restriction on the send buffer for blocking, non blocking and collective API has been lifted. It is permitted to access for read the send buffer while the operation is in progress."},"comment":{"url":"https://api.github.com/repos/wesbland/mpi-forum-historic/issues/comments/234067214","html_url":"https://github.com/wesbland/mpi-forum-historic/issues/45#issuecomment-234067214","issue_url":"https://api.github.com/repos/wesbland/mpi-forum-historic/issues/45","id":234067214,"user":{"login":"wesbland","id":1561207,"avatar_url":"https://avatars.githubusercontent.com/u/1561207?v=3","gravatar_id":"","url":"https://api.github.com/users/wesbland","html_url":"https://github.com/wesbland","followers_url":"https://api.github.com/users/wesbland/followers","following_url":"https://api.github.com/users/wesbland/following{/other_user}","gists_url":"https://api.github.com/users/wesbland/gists{/gist_id}","starred_url":"https://api.github.com/users/wesbland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wesbland/subscriptions","organizations_url":"https://api.github.com/users/wesbland/orgs","repos_url":"https://api.github.com/users/wesbland/repos","events_url":"https://api.github.com/users/wesbland/events{/privacy}","received_events_url":"https://api.github.com/users/wesbland/received_events","type":"User","site_admin":false},"created_at":"2016-07-20T20:11:14Z","updated_at":"2016-07-20T20:11:14Z","body":"**Originally by _alexander.supalov@intel.com_ on 2008-12-05 16:11:46 -0600**\n\n___\nBy the way of reviewing this proposal, let's consider the following scenario:\n \n- In the MPI_Isend, MPI maps the send buffer into the address space of the receiving process.\n- In the matching MPI_Recv, the receiving process makes a copy of the mapped send buffer into the receive buffer.\n- Once the copy is complete, the send buffer is mapped back into the sender address space during the wait/test call.\n \nWhat will happen one tries to access the send buffer in between?\n"}}
{"actor":{"display_login":"wesbland"}}