{"_id":"55286c7d391a362500d9b3f5","category":{"_id":"55284ed68962f339009a67e1","__v":6,"pages":["552854af60c60f230003fb96","5528553ad9e1db2d00cd9292","55286c7d391a362500d9b3f5","55290f5bceedaa0d00bc5c5b","56d1fb3d93f76e0b00bbc5e2","56d1fb6293f76e0b00bbc5e4"],"project":"552829408962f339009a678d","version":"552829408962f339009a6790","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2015-04-10T22:29:42.627Z","from_sync":false,"order":9,"slug":"protocol","title":"MTProto v2"},"parentDoc":null,"user":"55282916d9e1db2d00cd923c","version":{"_id":"552829408962f339009a6790","project":"552829408962f339009a678d","__v":26,"createdAt":"2015-04-10T19:49:20.516Z","releaseDate":"2015-04-10T19:49:20.516Z","categories":["552829418962f339009a6791","55284ed68962f339009a67e1","55286c73391a362500d9b3f4","552918f6b316811900149f59","5529b255d739240d00a3483e","553287590a578a0d008d4ff5","55329385e7d1fa0d003fc946","5550b55200420e0d00d1312f","55525fca953c9c0d00f507d7","559199695631432f002d358a","559d8d96980b801700d5ec7e","55c5e833cccdeb2d004e24b9","55d76504f662951900fc0e7d","55ea213cc62aa02f008229cd","56157b750f5ed00d00483dd8","561981fbac0924170069f4e8","561b8b1ea430930d0037ea67","563417428b86331700b488ca","56cd785bface161300dae0ec","56cdcc6e70db8a15006395f4","56cdf1b749abf10b0036a34a","56cedc8ce50c9c1b00830423","56e97ba8d825061900d1ac83","570d505228e6900e00477229","573614ca2ab52e1700c8e851","57d556a2496a3117004d70cf"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"project":"552829408962f339009a678d","__v":54,"updates":["55af1aaa555b900d0036d227","5837fa2fa746be0f00db654f","583be8cabc92770f0038ea69"],"next":{"pages":[],"description":""},"createdAt":"2015-04-11T00:36:13.994Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"auth":"required","params":[],"url":""},"isReference":false,"order":3,"body":"Transport Level is main and most complex part of protocol.\n\nMain goals of Transport Level\n  * Fast connection recreation\n  * Minimize traffic amount\n  * Connection-agnostic package transmition\n  * Recovering after connection die\n  * Recovering after server die\n  * Work in networks that work on buggy hardware that breaks connection in random situations\n\nIn reference protocol implementation only single active connection is used. This assumption makes implementation much more simpler and avoid some weird behaviour when new connection die and old one is still working. \n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Packages\"\n}\n[/block]\nEach transport level package is a structure\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"// Message Container\\nMessage {\\n\\t// message identifier\\n  messageId: long\\n  // message body\\n  message: byte\\n}\\n\\nPackage {\\n\\t// unique identifier that is constant thru all application lifetime \\n\\tauthId: long\\n  // random identifier of current session\\n  sessionId: long\\n  // message\\n  message: Message\\n}\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Requesting Auth Id\"\n}\n[/block]\nBefore sending and receiving packages it is required to perform ```authId``` calculation. In current revision ```authId``` is requested by simple special API request. In future version ```authId``` will be calculated by performing Diffie-Hellman key exchange. We assume that getting ```authId``` is implemented separately and is not part of main MTProto implementation.\n\nDuring getting ```authId``` authId/sessionId is required to be zero for all other operations authId and sessionId can't be zero.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"RequestAuthId {\\n\\tHEADER = 0xF0\\n}\\n\\nResponseAuthId {\\n\\tHEADER = 0xF1\\n  authId: long\\n}\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"RPC and Push Packages\"\n}\n[/block]\nClients can perform various RPC Requests and get RPC Responses. Server can send Push packages for notification client about updates.\n[block:callout]\n{\n  \"type\": \"info\",\n  \"body\": \"When implementing protocol it is useful to have ability to mark some RPC Requests as high priority and send them always as separate packages and before any other packages. This technic can reduce message send latency.\",\n  \"title\": \"Messages Priority\"\n}\n[/block]\n\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ProtoRpcRequest {\\n HEADER = 0x03\\n // Request body\\n payload: bytes\\n}\\n\\nProtoRpcResponse {\\n HEADER = 0x04\\n // messageId from Message that contains ProtoRpcRequest\\n messageId: long\\n // Response body\\n payload: bytes\\n}\\n\\nProtoPush {\\n HEADER = 0x05\\n // Push body\\n payload: bytes\\n}\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Service Packages\"\n}\n[/block]\nService packages are packages that used for keeping everything in sync.\n## Acknowledge\nReceiving of most messages need to be confirmed by receiver side. ProtoRpcResponse is automatically marking ProtoRpcRequest as confirmed.\nFor explicitly marking package as received used MessageAck (that doesn't need to be acknowledged).\nBehaviour of server side MessageAck and cleint side MessageAck is a bit different. Server try to send Acks as fast as it can and try to group all acks. \nClient send confirm:\n  * When new connection is created: first message is always MessageAck if available.\n  * When unconfirmed buffer contains more than 10 messages\n  * When client need to send some normal priority packages: client group it with MessageAck\n  * When uncorfirmed buffer is older than 1 minute\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"MessageAck {\\n HEADER = 0x06\\n // Message Identificators for confirmation\\n messageIds: longs\\n}\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n## Resend\nWhen sent message is not confirmed then other side can try to resend message or notify about other side about sent message. Usually this is performed when new connection is created or confirmation doesn't arrived in 1 minute.\n\nIf package is greater than 1kb peer can send notification about sent package for avoiding double-sending. When other peer gets notification about sent message it need to response with MessageAck or RequestResend depends on if peer received this message.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"// Notification about unsent message (usually ProtoRpcRequest or ProtoPush)\\nUnsentMessage {\\n HEADER = 0x07\\n // Sent Message Id\\n messageId: long\\n // Size of message in bytes\\n len: int\\n}\\n\\n// Notification about unsent ProtoRpcResponse\\nUnsentResponse {\\n HEADER = 0x08\\n // Sent Message Id\\n messageId: long\\n // Request Message Id\\n requestMessageId: long\\n // Size of message in bytes\\n len: int\\n}\\n \\n// Requesting resending of message\\nRequestResend {\\n HEADER = 0x09\\n // Message Id for resend\\n messageId: long\\n}\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n## NewSessionCreated\n**Most important part of protocol.** NewSessionCreated is client notification about session recreation because of performing Session Garbage Collection or because of Server Failure.\nWhen client receive NewSessionCreated it need to resend all ProtoRpcRequests, resubscribe to all events. All Session state is dropped when NewSessionCreated arrive. When client generate new session id then server send NewSessionCreated because there are never was a session with this id. NewSessionCreated are expected to appear randomly in any situation and client side code need to correctly implement it's support.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"NewSession {\\n HEADER = 0x0C\\n // Created Session Id\\n sessionId: long\\n // Message Id of Message that created session\\n messageId: long\\n}\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n## SessionHello\nNotification about session and auth id in session. Useful when connection is recreated, but there are no suitable data to send.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"SessionHello {\\n\\tHEADER = 0x0F\\n}\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n## SessionLost\nNotification about session of connection is lost. Client need to perform any request or send SessionHello package\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"SessionLost {\\n\\tHEADER = 0x10\\n}\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n## AuthIdInvalid\nClient's AuthId dies and connection is closed after sending this message\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"AuthIdInvalid {\\n\\tHEADER = 0x11\\n}\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n## Drop\nDrop is a package for notification about some problem with processing package. After sending Drop server close connection.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"Drop {\\n HEADER = 0x0D\\n // Message Id of message that causes Drop. May be zero if not available\\n messageId: long\\n // Error Message\\n errorMessage: String\\n}\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Container\"\n}\n[/block]\nFor grouping messages into containers protocol uses Container package\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"Container {\\n HEADER = 0x0A\\n // Messages count\\n count: varint\\n // Messages in container\\n data: Message[]\\n}\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]","excerpt":"Level for organization reliable package transmition","slug":"transport-level","type":"basic","title":"Transport Level"}

Transport Level

Level for organization reliable package transmition

Transport Level is main and most complex part of protocol. Main goals of Transport Level * Fast connection recreation * Minimize traffic amount * Connection-agnostic package transmition * Recovering after connection die * Recovering after server die * Work in networks that work on buggy hardware that breaks connection in random situations In reference protocol implementation only single active connection is used. This assumption makes implementation much more simpler and avoid some weird behaviour when new connection die and old one is still working. [block:api-header] { "type": "basic", "title": "Packages" } [/block] Each transport level package is a structure [block:code] { "codes": [ { "code": "// Message Container\nMessage {\n\t// message identifier\n messageId: long\n // message body\n message: byte\n}\n\nPackage {\n\t// unique identifier that is constant thru all application lifetime \n\tauthId: long\n // random identifier of current session\n sessionId: long\n // message\n message: Message\n}", "language": "text" } ] } [/block] [block:api-header] { "type": "basic", "title": "Requesting Auth Id" } [/block] Before sending and receiving packages it is required to perform ```authId``` calculation. In current revision ```authId``` is requested by simple special API request. In future version ```authId``` will be calculated by performing Diffie-Hellman key exchange. We assume that getting ```authId``` is implemented separately and is not part of main MTProto implementation. During getting ```authId``` authId/sessionId is required to be zero for all other operations authId and sessionId can't be zero. [block:code] { "codes": [ { "code": "RequestAuthId {\n\tHEADER = 0xF0\n}\n\nResponseAuthId {\n\tHEADER = 0xF1\n authId: long\n}", "language": "text" } ] } [/block] [block:api-header] { "type": "basic", "title": "RPC and Push Packages" } [/block] Clients can perform various RPC Requests and get RPC Responses. Server can send Push packages for notification client about updates. [block:callout] { "type": "info", "body": "When implementing protocol it is useful to have ability to mark some RPC Requests as high priority and send them always as separate packages and before any other packages. This technic can reduce message send latency.", "title": "Messages Priority" } [/block] [block:code] { "codes": [ { "code": "ProtoRpcRequest {\n HEADER = 0x03\n // Request body\n payload: bytes\n}\n\nProtoRpcResponse {\n HEADER = 0x04\n // messageId from Message that contains ProtoRpcRequest\n messageId: long\n // Response body\n payload: bytes\n}\n\nProtoPush {\n HEADER = 0x05\n // Push body\n payload: bytes\n}", "language": "text" } ] } [/block] [block:api-header] { "type": "basic", "title": "Service Packages" } [/block] Service packages are packages that used for keeping everything in sync. ## Acknowledge Receiving of most messages need to be confirmed by receiver side. ProtoRpcResponse is automatically marking ProtoRpcRequest as confirmed. For explicitly marking package as received used MessageAck (that doesn't need to be acknowledged). Behaviour of server side MessageAck and cleint side MessageAck is a bit different. Server try to send Acks as fast as it can and try to group all acks. Client send confirm: * When new connection is created: first message is always MessageAck if available. * When unconfirmed buffer contains more than 10 messages * When client need to send some normal priority packages: client group it with MessageAck * When uncorfirmed buffer is older than 1 minute [block:code] { "codes": [ { "code": "MessageAck {\n HEADER = 0x06\n // Message Identificators for confirmation\n messageIds: longs\n}", "language": "text" } ] } [/block] ## Resend When sent message is not confirmed then other side can try to resend message or notify about other side about sent message. Usually this is performed when new connection is created or confirmation doesn't arrived in 1 minute. If package is greater than 1kb peer can send notification about sent package for avoiding double-sending. When other peer gets notification about sent message it need to response with MessageAck or RequestResend depends on if peer received this message. [block:code] { "codes": [ { "code": "// Notification about unsent message (usually ProtoRpcRequest or ProtoPush)\nUnsentMessage {\n HEADER = 0x07\n // Sent Message Id\n messageId: long\n // Size of message in bytes\n len: int\n}\n\n// Notification about unsent ProtoRpcResponse\nUnsentResponse {\n HEADER = 0x08\n // Sent Message Id\n messageId: long\n // Request Message Id\n requestMessageId: long\n // Size of message in bytes\n len: int\n}\n \n// Requesting resending of message\nRequestResend {\n HEADER = 0x09\n // Message Id for resend\n messageId: long\n}", "language": "text" } ] } [/block] ## NewSessionCreated **Most important part of protocol.** NewSessionCreated is client notification about session recreation because of performing Session Garbage Collection or because of Server Failure. When client receive NewSessionCreated it need to resend all ProtoRpcRequests, resubscribe to all events. All Session state is dropped when NewSessionCreated arrive. When client generate new session id then server send NewSessionCreated because there are never was a session with this id. NewSessionCreated are expected to appear randomly in any situation and client side code need to correctly implement it's support. [block:code] { "codes": [ { "code": "NewSession {\n HEADER = 0x0C\n // Created Session Id\n sessionId: long\n // Message Id of Message that created session\n messageId: long\n}", "language": "text" } ] } [/block] ## SessionHello Notification about session and auth id in session. Useful when connection is recreated, but there are no suitable data to send. [block:code] { "codes": [ { "code": "SessionHello {\n\tHEADER = 0x0F\n}", "language": "text" } ] } [/block] ## SessionLost Notification about session of connection is lost. Client need to perform any request or send SessionHello package [block:code] { "codes": [ { "code": "SessionLost {\n\tHEADER = 0x10\n}", "language": "text" } ] } [/block] ## AuthIdInvalid Client's AuthId dies and connection is closed after sending this message [block:code] { "codes": [ { "code": "AuthIdInvalid {\n\tHEADER = 0x11\n}", "language": "text" } ] } [/block] ## Drop Drop is a package for notification about some problem with processing package. After sending Drop server close connection. [block:code] { "codes": [ { "code": "Drop {\n HEADER = 0x0D\n // Message Id of message that causes Drop. May be zero if not available\n messageId: long\n // Error Message\n errorMessage: String\n}", "language": "text" } ] } [/block] [block:api-header] { "type": "basic", "title": "Container" } [/block] For grouping messages into containers protocol uses Container package [block:code] { "codes": [ { "code": "Container {\n HEADER = 0x0A\n // Messages count\n count: varint\n // Messages in container\n data: Message[]\n}", "language": "text" } ] } [/block]