您是小站的第 35369 位访客,欢迎~

您现在的位置是: 网站首页 > 程序设计  > redis 

redis源码分析——3、简单动态字符串

2021年8月3日 23:33 361人围观

简介redis对C语言的原生char*做了封装,关于string只讨论两个问题:1、redis支持的最长string是多少?2、string的扩容策略

一、相关定义

  1. sds的定义
   typedef char *sds; 
  1. sdshdr的定义
   /* Note: sdshdr5 is never used, we just access the flags byte directly. 
    * However is here to document the layout of type 5 SDS strings. */ 
   struct __attribute__ ((__packed__)) sdshdr5 { 
       unsigned char flags; /* 3 lsb of type, and 5 msb of string length */ 
       char buf[]; 
   }; 
   struct __attribute__ ((__packed__)) sdshdr8 { 
       uint8_t len; /* used */ 
       uint8_t alloc; /* excluding the header and null terminator */ 
       unsigned char flags; /* 3 lsb of type, 5 unused bits */ 
       char buf[]; 
   }; 
   struct __attribute__ ((__packed__)) sdshdr16 { 
       uint16_t len; /* used */ 
       uint16_t alloc; /* excluding the header and null terminator */ 
       unsigned char flags; /* 3 lsb of type, 5 unused bits */ 
       char buf[]; 
   }; 
   struct __attribute__ ((__packed__)) sdshdr32 { 
       uint32_t len; /* used */ 
       uint32_t alloc; /* excluding the header and null terminator */ 
       unsigned char flags; /* 3 lsb of type, 5 unused bits */ 
       char buf[]; 
   }; 
   struct __attribute__ ((__packed__)) sdshdr64 { 
       uint64_t len; /* used */ 
       uint64_t alloc; /* excluding the header and null terminator */ 
       unsigned char flags; /* 3 lsb of type, 5 unused bits */ 
       char buf[]; 
   }; 
  1. 柔性数组

柔性数组成员(flexible array member)也叫伸缩性数组成员,这种代码结构产生于对动态结构体的需求。在日常的编程中,有时候需要在结构体中存放一个长度动态的字符串,一般的做法,是在结构体中定义一个指针成员,这个指针成员指向该字符串所在的动态内存空间,例如:

   struct s_test 
   { 
       int a; 
       double b; 
       char* p; 
   }; 

p指向字符串,这种方法造成字符串与结构体是分离的,不利于操作。把字符串和结构体连在一起的话,效果会更好,可以修改如下:

   char a[] = "Hello world"; 
   struct s_test *ptest = (struct s_test*)malloc(sizeof(s_test)+streln(a)+1); 
   strcpy(ptest+1,a); 

这样一来,(char*)(ptestt + 1)就是字符串“hello world”的地址。这时候p成了多余的东西,可以去掉。但是,又产生了另外一个问题:老是使用(char*)(ptestt + 1)不方便。如果能够找出一种方法,既能直接引用该字符串,又不占用结构体的空间,就完美了,符合这种条件的代码结构应该是一个非对象的符号地址,在结构体的尾部放置一个0长度的数组是一个绝妙的解决方案。不过,C/C++标准规定不能定义长度为0的数组,因此,有些编译器就把0长度的数组成员作为自己的非标准扩展,例如:

   struct s_test2 
   { 
       int a; 
       double b; 
       char c[0]; 
   }; 

c就叫柔性数组成员,如果把ptest指向的动态分配内存看作一个整体,c就是一个长度可以动态变化的结构体成员,柔性一词来源于此。c的长度为0,因此它不占用test的空间,同时ptest->c就是“hello world”的首地址,不需要再使用(char*)(ptestt + 1)这么丑陋的语法了。

二、redis string的最大长度

  1. 官网介绍
   Strings are the most basic kind of Redis value. Redis Strings are binary safe, this means that a Redis string can contain any kind of data, for instance a JPEG image or a serialized Ruby object. 

   A String value can be at max 512 Megabytes in length. 

那么就会有一个问题,既然最大是512Mb,那为什么还定义sdshdr64结构?

  1. processMultibulkBuffer中的定义
   int processMultibulkBuffer(client *c) { 
       char *newline = NULL; 
       int ok; 
       long long ll; 
       /****************省略代码**************/ 
        while(c->multibulklen) { 
            /****************省略代码**************/ 
            ok = string2ll(c->querybuf+c->qb_pos+1,newline-(c->querybuf+c->qb_pos+1),&ll); 
            if (!ok || ll < 0 || ll > server.proto_max_bulk_len) {  // 这里对长度做了判断 
                addReplyError(c,"Protocol error: invalid bulk length"); 
                setProtocolError("invalid bulk length",c); 
                return C_ERR; 
            } 
            /****************省略代码**************/ 
        } 
   }      
  1. proto-max-bulk-len配置
   createLongLongConfig("proto-max-bulk-len", NULL, MODIFIABLE_CONFIG, 0, LLONG_MAX, server.proto_max_bulk_len, 512ll*1024*1024, MEMORY_CONFIG, NULL, NULL), /* Bulk request max size */ 

三、sds扩容策略

  1. 当长度小于1Mb时,每次增长翻倍,当大于1Mb时,每次增长1Mb
   /* Enlarge the free space at the end of the sds string so that the caller 
    * is sure that after calling this function can overwrite up to addlen 
    * bytes after the end of the string, plus one more byte for nul term. 
    * 
    * Note: this does not change the *length* of the sds string as returned 
    * by sdslen(), but only the free buffer space we have. */ 
   sds sdsMakeRoomFor(sds s, size_t addlen) { 
       void *sh, *newsh; 
       size_t avail = sdsavail(s); 
       size_t len, newlen; 
       char type, oldtype = s[-1] & SDS_TYPE_MASK; 
       int hdrlen; 

       /* Return ASAP if there is enough space left. */ 
       if (avail >= addlen) return s; 

       len = sdslen(s); 
       sh = (char*)s-sdsHdrSize(oldtype); 
       newlen = (len+addlen); 
       if (newlen < SDS_MAX_PREALLOC) 
           newlen *= 2; // 翻倍 
       else 
           newlen += SDS_MAX_PREALLOC; // 增长1Mb 

       type = sdsReqType(newlen); 

       /* Don't use type 5: the user is appending to the string and type 5 is 
        * not able to remember empty space, so sdsMakeRoomFor() must be called 
        * at every appending operation. */ 
       if (type == SDS_TYPE_5) type = SDS_TYPE_8; 

       hdrlen = sdsHdrSize(type); 
       if (oldtype==type) { 
           newsh = s_realloc(sh, hdrlen+newlen+1); 
           if (newsh == NULL) return NULL; 
           s = (char*)newsh+hdrlen; 
       } else { 
           /* Since the header size changes, need to move the string forward, 
            * and can't use realloc */ 
           newsh = s_malloc(hdrlen+newlen+1); 
           if (newsh == NULL) return NULL; 
           memcpy((char*)newsh+hdrlen, s, len+1); 
           s_free(sh); 
           s = (char*)newsh+hdrlen; 
           s[-1] = type; 
           sdssetlen(s, len); 
       } 
       sdssetalloc(s, newlen); 
       return s; 
   }